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ABSTRACT 

This booklet contains information about the Human 
Interface Technology Laboratory (HITL) , which was established by the 
Washington Technology Center at the University of Washington to 
transform virtual world concepts and research into practical, 
economically viable technology products. The booklet is divided into 
seven sections: (l) a brief description of virtual worlds technology; 
(2) a des cription of HITL and how it interfaces with its parent 
institutions and surrounding community; (3) a list of HITL staff, 
fellows, and students, with a one-paragraph description of the 
activities of each; (4) a description of the Virtual Worlds 
Consortium, which works to enhance commercial applications of virtual 
worlds technology and a list of participating corporations; (5) lists 
of HITL core activities, enabling technologies, projected concept 
demonstration dates, and targeted markets; (6) an order form and a 
list of approximately 60 HITL technical publications; and (7) 
reprints from newspaper and magazine articles about virtual reality 
and HITL. A brief description of "sci .virtual-worlds , 11 a 
mul tidiscipl inary USENET newsgroup devoted to discussions of the 
subject, is given on the inside back cover. (KRN) 
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The Human Interface Technology Laboratory 
was established in October 1989 by the Washington 
Technology Centers to transform virtual world con- 
cepts and research into practical, economically 
viable technology products. 

The overall goal of the Laboratory is to 
develop a new generation of human-machine 
interfaces by: 

• Investigating and understanding the 
fundamentals of human perception and 
interaction with computers, information, 
and complex systems. 

• Pioneering new interface concepts 
focusing on virtual world technologies. 

• Creating and demonstrating new 
applications for virtual world technology 
in aerospace, medicine, education, design, 
entertainment, and related fields 

• Transferring advanced interface technology 
and research results to industry. 




Human Interface Technology Laboratory 
Washington Technology Centers 
at the 

University of Washington, FU-20 

Seattle, WA 98195 / 
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jm new interface archetype, 

that of the "virtual" inter- 

face, permits us to revise 
^^^^ both the way we think 

about computers and the 
way we think with them. These inter- 
faces allow panoramic presentations in 
three dimensions to be made to the 
eyes, ears, and hands of the user. 
"Virtual" images presented directly to 
the senses provide a participant with a 
total, global experience. The human 
literally wears the computer. The partic- 
ipant is surrounded by a "circumambience" 

of computer-synthesized information, a spatial "world" in which the person and computer 
can more effectively communicate. The operator interacts with this inclusive medium by 
looking at objects, pointing his or her hands, and giving verbal commands. The medium 
permits virtual objects, which appear to be real but which are virtual projections, to be 
touched and manipulated by the participant. 

To create the "virtual world" representation of information, the virtual-display hard- 
ware components are programmed with "mindware." Mindware is a special class of soft- 
ware that takes into account the perceptual organization of the human and dynamically 
creates the three-dimensional sound, video, and tactile images which surround the user. 

A virtual reality is not limited to a single person. Virtual worlds can be shared by two 
or more participants. Each participant has the ability to customize the rendering to his or 
her particular needs and preferences. With the addition of telecommunication links, these 
worlds can be shared by many participants and at different times, creating a new medium 
of communication: televirtuality. 

Virtual interfaces solve many existing interface problems and empower new and 
novel interfaces forteleoperation, computer-aided design, education, medical imaging, 
prostheses for the handicapped, and entertainment. 

Virtual interface technology provides a bold new opportunity for solving many of the 
perplexing problems of interfacing human and machine intelligences. With systematic 
development, virtual interfaces can be one of the greatest advances of our age, and a boon 
to industry. 
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The Human Interface Technology Laboratory was established in October 1989 
by the Washington Technology Center (WTC) to transform virtual world 
concepts and early research into practical, economically viable technology 
and products. The Laboratory performs basic research, develops interface 
technologies, and demonstrates applications for transfer to industry. 

Initially, the Laboratory's research and development program is focused on revolu- 
tionizing virtual interface technologies and creating new approaches to couple our senses, 
intellect, and psychomotor capabilities with complex machines. Ultimately, this activity will 
provide solutions to pervasive problems of information interface, in a variety of domains. 

The overall goal of the Laboratory is to develop a new generation of human-machine 
interfaces. This goal is expressed in the Laboratory's research agenda: 

To investigate and understand the fundamentals of human perception and 
interaction with the world, with computational machines, and with information 
systems. 

To pioneer new interface concepts focusing on virtual world technologies. 

To create and demonstrate new application areas for virtual worlds 
technology in aerospace, medicine, education, communication, design, and 
related fields. 

To transfer advanced interface technologies to the commercial sector. 

To provide an academic environment to train s>„jdents, disseminate research 
findings, and provide a general resource base for the development of 
human-computer interaction. 
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Washington Technology Centers 

The state legislature created the Washington Technology Center (WTC) in 1983 as 
a catalyst for statewide economic development. Its goal is to attract private industry and 
federal research dollars to help finance commercially promising research at the state's 
universities. The WTC marries research and private enterprise to produce new companies 
and high-tech products. The WTC is located in the new 67,000 square foot Fluke Hall. 

The WTC is producing technological developments in many areas including: ad- 
vanced materials, compound semiconductors, computer systems and software, veterinary 
and medical biotechnology, microsensors, and plant technology. The intellectual property 
associated with these technologies is in the form of patents, copyrights, trademarks, ser- 
vice marks, and trade names. The WTC's laboratories develop new prototypes and provide 
the kind of applied knowledge that can lead to breakthroughs for many high-tech industries. 



University of Washington 

The University of Washington is the premier research institution of the Pacific North- 
west, and a major contribi tor to research and development on a global scale. For the past 
20 years, the University has ranked among the top five institutions in the United States in 
terms of total federal grant and contract support. With an annual enrollment of more than 
33,000 students, the University provides a rich human resource base for academic re- 
search. The University ranks among the top ten in the United States in many fields, and 
most nudbly in the medical and engineering sciences. 



Seattle and Washington State 

Consistently related "America's No. 1 City," in terms of both livability and as a place 
to do business, Seattle is well-suited to the emerging global economy. The "Emerald City," 
located on the Puget Sound, has a mild climate with great scenic beauty, cozy neighbor- ' 
hoods, and proximity to natural areas. The region's burgeoning cultural scene and high- 
tech industry are making their mark on the Pacific Rim and beyond. The local scene is 
informal and friendly. 

The State of Washington offers a unique blend of urban, suburban, rural, and wilder- 
ness regions, with lifestyles to match. Its economy, balanced among natural resources 
agnculture, crafts, and high technology design and manufacturing, continues to expand' 
Washington's progressive politics has produced an unusual alliance between government 
and business that ensures the state's continuing development into the 21st Century 



9 

ERIC 



G 

Human Interface Technology Laboratory 



Virtual Worlds 
C ON SO RTIUM 



• Alias Research, Inc. 

• American Express Company 

• The Boeing Company 

• Digital Equipment Corporation 

• Division Ltd. 

• Ford Motor Company 

• Fujitsu Research Institute 

• Franz, Inc. 

• Insight Inc. 

• Kopin Corporation 

• Kubota Pacific Computer, Inc. 

• Fluke Manufacturing Co., Inc. 

• Microsoft Corporation 

• Port of Seattle 

• Sharp Corporation 

• Stratos 

• Sun Microsystems 

• US Navy 

• US WEST Communications 

• Virtual Reality Inc. 



Human Interface Technology Laboratory 
of the Washington Technology Center 
at the University of Washington 
Seattle, WA 
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rhe Virtual Worlds Consortium supports the Human Interface Technology 
Laboratory as a forum for the advancement of virtual worlds technology; for 
educating professionals, students, and the general public regarding virtual 
worlds technology; and for enhancing commercial applications of virtual 
worlds technology. 

The Virtual Worlds Consortium brings together industry, government, and academia 
to turn virtual world concepts into marketable technology. As a Member of the Consortium 
you will gain a strategic posture in promoting the development of an emerging technology. 
More specifically, you will: 

Help guide the research and development agenda of the Human Interface 
Technology Laboratory 

Be In touch with emerging research through two working meetings each 
year 



Receive a semi-annual Consortium newsletter, timed for publication 
between working meetings, that includes news of the field, research-in- 
progress, and other items of interest 

Have prepublication access to the Laboratory's research findings 

Be able to recruit from our corps of graduate students who are directly 
involved in the development of virtual worlds technology 

Pinpoint areas of research specific to your strategic needs and contract 
with the Laboratory for their investigation 

Be informed of technological accomplishments of a non-proprietary 
nature as they occur, for possible commercialization of inventions and 
novel processes 

Be in the company of other organizations and professionals similarly 
interested in the emerging field of virtual worlds technology 

Work side-by-side with Laboratory personnel scientists and engineers as 
Industry Fellows and Associates 



Membership in the Virtual Worlds Consortium costs $50,000 per year. Annual 
sponsorship fees are pooled to support selected research projects at the Laboratory. 
Because Members have access to the research and development of virtual worlds technol- 
ogy at its initial stages, they enjoy a competitive advantage as they transfer VR applications 
into the commercial environment. 
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Thomas A. Furness III, Ph.D v Director. Dr. Furness brings to the Human Interface Technology 
Laboratory 23 years of virtual world research experience with the U.S. Air Force. Prior to founding the 
Laboratory, Dr. Furness was Chief of the Visual Display Systems Branch, Human Engineering Division of 
the Armstrong Aerospace Medical Research Laboratory (AAMRL) at Wright-Patterson Air Force Base, 
Ohio. His staff of 50 government and contractor scientists, engineers, and technicians pioneered advanced 
interface concepts for fighter aircraft including the Super-Cockpit, a virtual cockpit that the pilot wears. 
This system created a three-dimensional visual, aural, and tactile world enabling pilots to operate 
complex aircraft with natural hand and eye movements and voice control. 

Dr. Furness is a Professor in the Department of Industrial Engineering at the University of 
Washington, where he conducts classes in human factors and the design of virtual world technology. Dr. 
Furness holds a Ph.D. in Engineering and Applied Science form the University of Southampton, England. 

Colin Bricken, Software Engineer. Mr. Bricken received his B.A. in General Art from the 
University of Washington in June, 1992. His current project involves implementing the Universal 
Motivator, an evolving interface to V.R. that includes a V.R. database and dynamics specification, 3-D 
light and sound rendering, and midi drivers. 

William Bricken, Ph.D. , Principal Scientist Dr. Bricken was formerly Director of the Autodesk 
Research Laboratory, where he led the development of Cyberspace, Autodesk's CAD application of virtual 
world technology. He was also Principal Scientist at ADS, where he pioneered the development of high 
performance inference engines, visual programming languages, and intructable interfaces. Dr. Bricken holds 
a Ph.D. in Research Methodology and an M.S. in Statistics from Stanford University, and a Dip.Ed. from 
Monalsh Unviersity. He has taught at all levels, including principal of an innovative primary school and 
an Assistant Professor of Social Psychology of Education. 

Robert Burstein, Research Engineer. Mr. Burstein manages the Laboratory facilites and plays a 
major role in the fabrication of virtual-worlds presentation hardware. He holds a B.S.E.E. from the 
University of Dayton and has contributed to other pioneering WTC enterprises, including most recently the 
Materials Fabrication Laboratory. Mr. Burstein is a trained musician who works on synthetic generation of 
three-dimensional sound. 

Marc Cygnus, Research Engineer. Mr. Cygnus received his B.S. degree in Computer Science from the 
University of Delaware in 1991. He is currently pursuing his M.S. in Industrial Engineering (Human 
Factors) and working on biomedical projects involving the development of input devices utilizing 
biopotentials at the HIT Lab. 

Ann Elias, Program Support Supervisor. Ms. Elias coordinates the Laboratory's administrative 
activities. She has worked with local high-tech firms in the Seattle vicinity, been involved in Northwest 
politics as a campaign manager, and is an active fundraiser for the arts community. Ms. Elias holds a B.S. 
from Oregon State University. 

Alden Jones, Program Assistant Ms. Jones received a B.A. in Art History and English from the 
University of California at Santa Barbara and then worked in the design and museum fields and the film 
industry. She coordinated the Lab's most recent Industry Symposium on Virtual Worlds Technology and is 
currently working on the Lab's next issue of the HIT Lab Review. 

Arthur Kerr, Business Manager. Mr. Kerr received his B.S. in Mathematics and Mechanical 
Engineering from the United States Air Force Academy in Colorado, his M.B.A. from Harvard University 
Graduate School of Business Administration and completed post-graduate studies at the Royal College of 
Defense Studies in London. Prior to his work at the HIT Lab, Mr. Kerr directed national-level 
communications programs in the Air Force, and supervised international sales in Cairo, Egypt. He also 
taught on the faculty of the Air Force Academy and managed development, test and evaluation activities 
on the Airborne Warning and Control System (AW ACS) in Seattle. 



Brian Karr, Research Engineer, Mr. Karr has a B.S. from Evergreen State College. His degree is a 
combination of Physics and Computer Science. Mr. Karr has two years of graduate studies in the E.E. Ph.D. 
program at the University of Washington. His past VR experience includes designing and building a system 
that converts motion into musical instrument contol data (MIDI). He is presently researching 3-D (spatial) 
sound and developing applications for demonstration and res *arch for inclusion in virtual worlds. 

Joel Kollin, Research Engineer, Optical Systems- Mr. Kollin is responsible for the development of 
new Virtual Worlds display technology as well as other optical systems at the Lab. He holds a M.S. from 
the MIT Media Laboratory, where he pioneered research on Holographic Television, and a B.S.E. 
(Electrical Engineering) from the University of Michigan. At KMS Fusion, he worked on interferometry 
systems for laser-plasma interaction experiments and conducted design studies for a Real-Time 
Holographic Stereogram Display. Mr. Kollin has also worked on short-term projects for the Polaroid Corp. 
and the Industrial Technology Institute. 

Andy MacDonald, Research Engineer. Mr. MacDonald is the primary systems administrator at the 
HIT Lab. He graduated from M.I.T. in 1988 with a degree in Mathematics and Computer Science and his 
current interests include distributed computing and artificial life. 

Kymerie Riggs, Administrator, Virtual Worlds Consortium/Administrative Assistant Ms. Riggs is 
currently administrator of the Virtual Worlds Consortium, a corporate enterprise committed to the research 
and development of virtual worlds technology. Her other duties involve proposal/funding support, 
administrative details, public relations, and coordination of the graduate and undergraduate student 
activities at the Lab. Ms. Riggs has a B.A. in English and a B.A. in Communications from the University of 
Washington. 

Suzanne Weghorst, Research Scientist Ms. Weghorst brings a combination of expertise in computer 
science and behavioral science to her work. Her most recent project, with the Department of Radiology at 
the University of Washington Medical Center, evaluated the clinical acceptability of a comprehensive 
computer system for acquiring, storing, and displaying digital images. She holds an M.A. in Psychology 
from the University of California at Riverside and an M.S. in Computer Science at the University of 
Washington. Her thesis examined user perception of graphical interface displays. 

Industrial Fellows 

Ben Childers, Orbis International. Mr. Childers received a B.S. in Environmental Design from 
Texas A&M University in August of 1990. During the past two years he has been involved with Orbis 
International in the design of their new DC-10 Flying Eye Hospital. He is now using graphic imaging and 
animation, which was used to help in the design of ihe DC-10, to help design an eye surgery simulator and 
teaching aide to be deployed on the DC-10. 

Bernard Ulozas, Ph.D, US Navy, Training Specialist, Navy Personel R&D Center. Dr. Ulozas 
received a Doctor of Arts degree in History and Curriculum Development from Carnegie-Mellon University 
in 1980. He began work as an Education Specialist with the U.S. Navy at Service School Command, Great 
Lakes, Illinois and managed the Curriculum and Instructional Standards Office. He later established and 
directed the Training Department at the Naval Ship Systems Engineering Station in Philadelphia, PA. 
Presently Dr. Ulozas is working at the Navy Personnel Research and Development Center in San Diego, CA 
and is an industrial fellow at the HIT Lab. His research efforts include development of interactive 
courseware, intelligent tutoring systems and virtual environments for the U.S. Navy Damage Control 
personnel. 
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Chris Byrne, Research Associate. Ms. Byrne received a B.S. in Industrial Engineering and 
an M.S.E. in Design of Experiments and Quality Control from the University of Washington. She 
spent four years working for US West and is currently pursuing her Ph.D. in Industrial Engineering. 
Her interests include using virtual reality for holistic solutions to engineering problems. Ms. Byrne 
is working on the development of the education project at the HIT Lab. 

Geoff Coco, Research Assistant Mr. Coco received a B.S. in Computer Science from 
Dartmouth College and then worked in tne field of Graphics Application for Apple Computers. He 
is currently pursuing an M.S.E. (Human-Computer Interaction) at the University of Washington 
while working at the HIT Lab designing the Virtual Environment Operating System. 

Paul Damct, Research Assistant Mr. Danset graduated in 1981 with a B.S. in Physics and 
is currently a graduate student in the Electrical Engineering Department at the University of 
Washington. He has ten years work experience in software and hardware, primarily in a research 
environment. His research interests include signal processing and pattern recognition applications 
to VR. 

Fernando Diaz, Student Aide, Mr. Diaz completed a Computer Information Systems 
curriculum at the University of Alaska in 1984. Additional studies included cognitive psychology, 
philosophy, management information systems. Mr. Diaz completed an engineering internship at 
General Electric Aerospace Division and was System Administrator at the Sony Advanced Video 
Technology Center. Currently, Mr. Diaz is pursuing a B.S. in Computer Science & Cognitive Science 
at the University of Washington. His degree program includes research in software tools for 
virtual world design at the HIT lab. 

Toni Emerson, Lab Associate, Ms Emerson has a B.A. in Drama and a B.A. in Spanish 
Literature from the University of Washington. She expects to receive a Masters of Library Science 
in the Fall of 1992, and plans to pursue a Ph.D in Library and Information Science. Ms. Emerson is 
our HITL "Cybrarian" and high-tech information gatherer on the internet and various commercial 
databases. She wants to specialize in the creation of multi-media information systems (specificaly 
virtual information systems) and hopes that her research will help to empower the end-user 

Daniel Henry, Research Assistant Mr. Henry received a B.A. in Architecture from the 
University of Berkeley focusing primarily on the understanding of human behavior in space design. 
For several years he developed interactive computer animation for kiosks and expositions. Mr. 
Henry's work in the Laboratory involves helping enrich the existing Virtual Worlds usinp Swivel 
3-D for modeling and Body Electric for interactive programming. 

Ari Hollander, Research Assistant, Mr. Hollander received his B.A. in Astrophysics from 
the University of California at Berkeley in 1991. He is currently pursuing his M.S. in Industrial 
Engineering at the University of Washington and working on position sensing at the HIT Lab. 

Jeff James, Research Assistant Mr. James received a B.S. in Psychology and a B.S. in 
Electrical Engineering from the University of Illinois at Urbana-Champaign. He is pursuring his 
M.S.I.E. and working on extending boundary mathematics into visual programming. 

Karen Jones, Research Assistant Ms. Jones has a B.S. in Industrial Enigeering from the 
University of Washington and is currently in the M.S. Engineering program. She has worked on 
evaluating eye tracking devices for the laboratory for further research and is working on 
developing manufacturing applications for VR. 



Dav Lion, Research Assistant Mr. Lion attended Brown University where he majored in 
both Studio Art and Computer Science. He helped design and is now currently writing the Virtual 
Environment Operating System for the HIT Lab. Mr. Lion is also pursuing an M.S.E. (Human- 
Computer Interaction) the University of Washington. 

Max Minkoff, Research Assistant Mr. Minkoff received an Information Systems degree 
from Drexel University in Philadelphia and spent two years working in industry on microcomputer 
user support. He is currently pursuing his M.S. in Industrial Engineering at the University of 
Washington. Mr. Minkoff is developing world-building tools for the Virtual Environment 
Operating System at the HIT Lab. 

Daniel Pezely, Lab Associate. Mr. Pezely completed a B.S. in Computer Science with a 
Philosophy minor at the University of Delaware and has worked in the computer industry as a 
programmer and consultant since the age of 15. He attributes his CAD experience, his fascination 
for telecommunications and his artistic abilities to his help in developing the Virtual Environment 
Operating System. 

Daniel Pirone, Lab Associate, Mr. Pirone has a B.S. in Computer Science from the 
University of Delaware and has worked in the field of Artificial Life. After working in a robotics 
lab at Hewlett Packard, he was invited to study at the Santa Fe Institute. He is planning to pursue 
an M.S.E. (Human Computer Interaction) at the University of Washington while working at the 
HIT Lab designing tools for the Virtual Environment Operation System. 

Jerry Prothero, Research Assistant Mr. Prothero received a B.S. in Physics and Computer 
Science from the University of Washington. He expects to receive his M.S. in Industrial 
Engineering in 1993. His projects at the lab include 3D graphics and bio-medical visualization. 

Jesus Savage, Research Associate. Mr. Savage graduated in 1984 with a B.S. in Computer 
Engineering and a M.S. in Electrical Engineering in 1990 from the National Autonomous University 
of Mexico. Mr. Savage, a Fulbright Scholar, is currently pursuing his Ph.D. in Electrical 
Engineering at the University of Washington. His project involves researching voice recognition for 
inclusion in virtual worlds. 

Mark Takacs, Research Assistant Mr. Takacs received his B.S. in Computer Science from 
Miami University in Oxford, OH in 1991 and is currently pursuing his M.S. in Industrial 
Engineering. His interests include multiple participant virtual worlds and body icons. 

Kevin Welton, Research Associate. In 1977, Mr. Welton graduated summa cum laude from 
U. of California's Davis campus with a B.S. in Agricultural Engineering. Next came a 14 year stint 
with Weyerhaeuser conducting research and doing prototype equipment development to improve 
log and lumber process systems and equipment. His primary roles covered that of project manager, 
staff engineer, and computing specialist; his primary technical foci included EE R&D and design, 
and operations research. Mr. Welton is now working towards a PhD in EE. At the Hl i Lab, his focus 
is on Head Up Display hardware, and on how to improve the user's perception that the viewed 
virtual objects are part of the real world. 

Ryoko Williamson, Student Aide. Ms. Williamson expects to receive an interdisciplinary 
B.S. degree in Cognitive Science with an emphasis on computer-human interface in December 1992. 
Her research interests at the HIT lab include human factors issues and cognitive aspects of V.R. 
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HTTL TECHNICAL PUBLICATIONS 



The Human Interface Technology Laboratory has produced a list of technical publications 
that are available to the public for a nominal fee. All publications are authored by KIT Lab 
staff, with the exception of the video // Welcome to Virtual Reality/' which was produced by 
US West Communications Corporate Television Division. The cost for each are as follows: 



Memorandums $5 .00 

Reports $10.00 

Presentations $10.00 

Videos as noted 

To order any of the following publications, please fill out the order form below and mail 
with check payable to the Human Interface Technology Laboratory to: 

Kymerie Riggs 

Human Interface Technology Laboratory 
University of Washington, FJ-15 
Seattle, WA 98195 

Please note: Publications will not be mailed until check has been received 



HITL Technical Publications Order Form 

Name: 

Company: 

Address: 

City/State/Zip: ■ 
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Virtual reality transforms data into 
things you can see, feel, and even 
touch. The implications for new 
insights and new perspectives are huge. 

—Christopher Barr [1] 

Virtual reality (VR) has become a 
buzzword in the media of late. For 
those who may not have heard about 
it, VR involves technologies that 
promise to revolutionize the human- 
computer interface, as well as the way 
we search, display, and assimilate the 
information we have stored in 
databases. 

By donning VR's "goggles-and- 
glove" hardware, you enter a three- 
dimensional world generated from 
real-time computer graphics. The 
goggles or head-mounted displays 
(HMDs) contain two, tiny video mon- 
itors that immerse you in a 3-D virtual 
world. The glove allows you to use 
natural, pointing gestures to navigate 
your way through this virtual world, 
manipulate objects, and receive tactile 
feedback. Earphones in the HMD 
transmit three-dimensional sound. 
While state-of-the-a"t VR hardware 
looks a bit like a S.C.U.B.A. mask and 
glove, the trend for the future will be 
hardware that is less intrusive. 

Although VR is still in its infancy, it 
is likely to become the ultimate 
multimedia computing experience. 
Virtual reality game arcades already 
exist. VR's potential in the areas of 
architectural design, medical imaging, 
and scientific visualization are gaining 
acceptance. The benefits of being able 
to walk through a building and change 
things during the design phase, 
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superimpose an X-ray over a patient in 
surgery, or manipulate molecular 
models when testing drugs are 
obvious to people working in those 
areas. You can do all of these things 
and much more in virtual reality. 

I first heard about virtual reality a 
few years ago from Steve Cisler, Apple 
Computer Senior Scientist and former 
Mac Monitor columnist tor this 
journal. He mentioned that a labora- 
tory to develop this technology had 
opened up at the University of 
Washington, which is literally in my 
backyard. When Steve described how 
you could move through a virtual 
world using your own gestures, I 
dismissed the technology outright. 
Since I am one of those people who 
gets lost easily in the real world, I 
assumed I would get lost just as easily 
in a virtual world. This sounded like a 
technology with limited appeal, at 
least for those of us who don't have a 
high degree of spatial and visual 
awareness. 

WHAT CAN WE LEARN FROM 
ARTISTS? 

I ignored most of the VR media 
hype until the spring of 1992 when I 
heard Dr. Thomas A. Fumess III speak 
to a group of special librarians. Dr. 
Fumess is Director of the Human 
Interface Technology Laboratory (HIT 
Lab), the lab that Steve Cisler had told 
me about a few vears earlier. Dr. 
Fumess' lecture had an intriguing title, 
"Virtual Museums. Knowbots and the 
End of Steelcase/' and his enthusiastic 
delivery surpassed my expectations. 

In addition to showing slides of the 
high-tech world of virtual reality. Dr. 



Fumess incorporated slides of his own 
portrait drawings into his presenta- 
tion. The drawings were from a class 
he and his wife had taken which was 
based on the principles of Bettv 
Edwards' book. Drawing on the Right 
Side of the Braii : 

By studying this book, you will 
team how to see. That is. you will 
learn how to process visual infor- 
mation in the special way used by 
artists. Hint 'way is different trom the 
way you usually process visual 
information and seems to require that 
you use your brain in a different zlyiu 
than you ordinarily use it (2 J. 

If you are one of those people who 
thinks he can't draw, buy this book. 
Don't just read about the exercises. Tnt 
them. After just a few exercises you 
will probably realize you are using a 
different part of your brain, a part that 
you don't ordinarily tap into. I guaran- 
tee you will be amazed at how these 
exercises can change your perceptions 
and expand your awareness, not to 
mention teach you how to draw. 

THE ORIGINS OF VIRTUAL 
REALITY 

The premise of virtual interlace 
technology at its most basic level 
seems obvious. We live in a three- 
dimensional world. However, in order 
to convey information about this 
world, we translate it into two- 
dimensions such as words and 
symbols. Then we have to translate 
this 2-D information back into 3-D, at 
least in our own minds, in order to use 
it in the real world. Why not speed up 
the learning process by eliminating the 
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intermediate 2-D translation wherever 
possible? 

Tins i«. a radical idea, particularly tor 
U>lk> in the information industry who 
tend to bo verbally-oriented. We have 
become ^o accustomed to poring over 
hundreds nf pago ot downloaded text 
and numbers to clean a tew relevant 
tacts, that we torcct that only the most 
highlv motivated have the inclination 
or patience to gather inrormation in 
this way. The obstacle to selling data to 
the end-user is not merely the inter- 
face. The presentation ot the data 
could do with some sprucing up. 

However, there is a tar more impor- 
tant and pressing issue here than a 
convenient interface tor accessing 
information or displaying it in a pretty 
package. Who among us can deny that 
we have reached the point of infor- 
mation overload? Granted, with all of 
this information literally at our 
fingertips, participants in the online 
information industry might be the worst 
of the information junkies. Am I alone 
when I say I read more than almost 
anyone I know? Just reading e-mail can 
become a full-time job if you're not 
careful The point is. we have just about 
reached the limit of the amount of 
information we can assimilate unless ire 
cfiange the uw av think. 

User interfaces and computer tech- 
nology . . . are van .in evolutionary 
process. Designs that :ivrk change the 
way zee think. 

— Don Sornian [41 

. This was preciselv the problem Dr. 
Furness was raced with solving when 
he was director ot the Super Cockpit 
program at Wright- Patterson Air Force 
Base, where he worked prior to 
starting the HIT Lab. The amount of 
information fighter pilots had to 
assimilate from the cockpit's radar, 
instruments, and command communi- 
cations had become too much. The 
slightest error by a pilot who was 
fighting G forces while flying one of 
these expensive machines through 
enemy territory could make for a very 
bad day! The solution was a virtual 
cockpit that gives the pilot back his 
three-dimensional capability; it uses 
audio, visual, and tactile feedback to 
transmit information to him. 
Experienced pilots loved it, and it 
turned out to be an incredibly fast and 
efficient means of teaching new pilots 



to fly. This was the first application of 
virtual interface technology. 

VIRTUAL REALITY AND ONUNE 

So what does virtual reality have to 
do with the online industry? Virtual 
interface technology may well be the 
means by which we interact with 
databases in the future. Instead of 
being limited to a keyboard that we 
use to "dialog" with the computer, i.e., 
type in commands, keywords, 
numbers or menu options, our body 
and all or most of its senses may be the 
central component of the interface. 

/ believe that conversation is the wrong 
model for dealing ivith a computer — a 
model that misleads inexperienced 
users and invites even experienced 
software designers to build hard-to-use 
systems. When you are interacting 
with a computer, you are not 
conversing with another person. You 
are exploring another ivorld. 

—]ohn Walker 13] 

Virtual worlds technology may 
provide a key to assimilating complex 
information stored in databases. 
Instead of displaying the results of a 
database search on a screen, we may 
be able to step "through the looking 
glass" [31 and immerse ourselves in 
the data. This presentation of data 
could take advantage of all of our 
senses. 

FROM SKEPTIC TO TRUE 
BELIEVER 

At the end of Dr. Furness' lecture, I 
asked a question. Since most of the 
applications of this technology that I 
had heard about seemed to have 
something to do with manipulating 
physical reality, I wondered if there 
were other types of applications. 
Being a business information 
specialist. I specifically asked if there 
were applications that someone like a 
stockbroker or accountant might use. 

Dr. Furness gave an example that I 
have since come across in the litera- 
ture. I think it is a particularly good 
example and warrants repeating. He 
explained that virtual reality is well- 
suited to representing complex 
numeric data. For example, you could 
portray the stock market as a wheat 
field, with each stalk of wheat rep- 
resenting a different stock. Price 
fluctuations or other factors could be 



portrayed by the stalks growing and 
shrinking. You could literally walk 
through this virtual field of real-time 
stock data. 

Do any of you who read the stock 
market pages or search stock market 
data in online databases agree that 
this is an amazing application? 
Personally, I follow about fifteen to 
twenty stocks. That's really about my 
limit. However, if I had a 360-degree 
view of this data represented 
pictorially and in three dimensions, I 
am certain I could follow many more. 
Obviously, this type of representation 
would also'make it much easier to 
identify patterns and trends. While 
some would argue that you can do 
almost the same thing on a computer 
screen, I take exception to that I think 
that the 360-degree field of vision and 
being able to interact with the data in 
a three-dimensional virtual world 
make all the difference. 

This example made me a convert. I 
do believe this technology has enor- 
mous potential to change the way we 
think and learn. I also think it has 
enormous implications for the online 
industry. We are the keepers of huge 
amounts of data, which will be crucial 
information In virtual worlds. 

This example also prompted me to 
request an interview with Dr. Furness. 
Read on to find out about the 
activities of the HIT Lab, how online 
information companies can participate 
in this emerging industry, and what it 
all may mean for the future of online 
databases. 
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>: . -• . ONLINE INTERVIEWS 
, DR. THOMAS A. FURNES S HI, 

VIRTUAL REALITY PIONEER 




"VSXDemo" is a 
project the HIT Lab 
did in conjunction 
with Boeing 
Computer Services. 



(Author's Note: This is a recorded inter- 
view with Dr. Thomas A, Furness III, 
Director of the Human Interface 
Technology Laboratory, a division of the 
Washington Technology Center located on 
the University of Washington (Seattle, 
WA) campus. This interview took place 
on May 27, 2991— CM) 

CLM: Let's begin with the terms 
"virtual reality," "virtual world," and 
"virtual interface." Could you give 
brief definitions? 

TAF: Sure. First of all, let me explain 
what a "virtual image" is. "Virtual" 
comes from a term in optical physics 
that describes the type of image that 
you see from a particular location in 
space where there is no object in that 
location. For example, when you see 
a reflection in a mirror, you see a 
person that's on the other side of the 



glass looking through, but there is no 
actual person there on that other side 
of the glass. That is what we call a 
virtual image. 



Virtual reality is an 
environment that you 
create using a 
combination of visual, 
auditory and tactile 
images... 



What can be created with virtual 
displays are not only visual, but 
auditory and tactile images— things 
that you see, things that you hear, 
things that you feel — that appear to be 
coming from a location in space, but 



aren't really there. The beauty of this is 
that with image projectors you can 
project this into your eye so that you 
can surround yourself with these 
images without taking up a lot of space. 
Ifs projected in 3-D; you have separate 
images going into each eye, which 
gives you a stereographic-type displav. 

Virtual reality is an environment that 
you create using a combination of 
visual, auditory and tactile images so 
that it becomes an alternative, sort of an 
artificial, environment or reality. We call 
it a "reality" because you perceive it as if 
it is a world. Ifs just like you're walking 
into another world, and you're per- 
ceiving it as if it becomes reality itself. 

CLM: Is "virtual reality" synonymous 
with "virtual world"? 

TAF: Well, it's very difficult to denne 
all of these terms, but we've defined 
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what a virtual image is. The world that 
vou create or what you perceive is the 
"medium— the virtual medium that 
creates virtual images tor your eyes 
and ears and hands. But then the 
message that is in that medium is what 
we call an "environment/' e.g., a room 
with chairs and tables and objects in it. 
Now, in that regard, "virtual environ- 
ment" and "virtual reality" are the 
same thing. The "virtual world" is the 
model that you are playing inside of. 
So, basically, "virtual world/' "virtual 
reality" and "virtual environment" are 
the same thing. 

Then there's another term called 
"cyberspace." We define cyberspace 
as a network of people who are 
linked together to form a virtual 
community. This is where people 
come together in a virtual place, like 
a real place, only it doesn't exist 
anywhere. Yet all these people from 
different geographical locations are 
coming into this one space and 
interacting with each other as if it 
were a real place. This is what we call 
a "virtual common," and the linking 



of people together this way is 
"cyberspace." 

CLM: Dr. Furness, would you give me 
a brief summary of your background? 

TAF: After graduating from Duke 
University in Electrical Engineering in 
1966, I started working for the 
Department of Defense, Department of 
the Air Force at Wright-Patterson Air 
Force Base. I was an officer in the Air 
Force for five years, and then I 
converted to civil service. I worked at 
Wright-Patterson for 23 years. 

It was there that I became involved 
with virtual interfaces, primarily 
trying to solve problems of how 
humans interact with very complex 
machines, xn this case, f r^iter airplane 
cockpits. I was trying to build a 
cockpit that the pilot wears, that would 
directly input information into the 
senses — visual, auditory, and tactile 
senses — thus enabling the pilot to 
more effectively interact with the 
world. This was a three-dimensional 
presentation that took advantage of 



the way humans are organized. I 
worked on this technology until 1989. 

CLM: How did you make the transition 
to the Human Interface Technology 
(HIT) Lab? 

TAF: In 1986 the Air Force asked me to 
hold a news conference about the 
work we were doing in VCASS 
(Visually-Coupled Airborne System 
Simulator) on the virtual cockpit. This 
was about the time that the Navy had 
these $800 toilet seats and 5500 
hammers, and the military needed 
something good to relate. I guess they 
felt that our work would help out. We 
did a news release and David Martin, 
the CBS Pentagon correspondent for 
Dan Rather, spent all day in our lab, 
and we ended up on the "CBS Evening 
News." After the segment we were 
inundated with calls from the press. 
NBC came in, and then ABC, BBC, 
CBC, and Australian television and 
New Zealand television. Nova came in 
and did 'Top Gun and Beyond." Then 
came Newsweek and U.S. News and 




'Virtual Seattle" (top left) 
demonstrates the viewing and 
movement capabilities of virtual reality, 
allowing both flying and instantaneous 
changes of perspective. 



"Planetscape" (bottom right) is a 
virtual world created by children at the 
Pacific "-ience Center's Technology 
Dayc ring the Summer of 1991 . 
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World Report, the New York Times, 
Washington Post, all the trade journals. 
We didn't get any research done after 
that. We just did show business. 

I also started getting phone calls 
from people around the country who 
had seen these programs on tele- 
vision. I got a call from a mother 
whose child had cerebral palsy, and 
she asked if there was anything this 
technology could do to help her 
child. I got a call from a surgeon who 
was trying to perfect a new surgical 
procedure, and he wanted to be on 
the inside looking out, rather than on 
the outside looking in. Another 
surgeon, a thoracic surgeon, was 
interested in how he could project the 
X-ray into the patient and use that as 
a navigational system to find his way 
around the pat 5 2nt. A firefighting 
company called and wanted to know 
how this technology could be used to 
build a display for firefighters to use 
when they were goi.*g into burning 
buildings to show where the firemen 
were located so that the fire chief on 
the outside could direct them. An 
anesthesiologist called and wanted to 
know how he could use the tech- 
nology to monitor patients and see all 
the vital signs and other information 
concurrently. In answer to all of those 
questions, I said, "Well, yes, you can 
do that. As a matter of fact, it would be 
sort of easy compared with what I've 
been working on!" 



laboratory which would transfer the 
technologies we had been working on 
in the DoD to the private sector where 
they could make a difference. 

I put together a plan and shopped 
around the country for a place to put 
this lab. I went to a number of 
universities including MIT, Carnegie 
Mellon, University of Texas, University 
of Utah, Berkeley, Stanford, and so 
forth, and ended up here at the 
University of Washington. I came to 
Washington because I was given the 
opportunity to work on the technology 
from an academic side as a tenured 
faculty member in the College of 
Engineering; and from a business side 
as a director of a new laboratory that 
was part of the Washington Tech- 
nology Center. The Center is an 
industry-oriented laboratory and ifs 
ecumenical— it isn't associated with or 
locked into one department. That 
means it can be an interdisciplinary 
program with a mandate to build from 
a multifaceted agenda. That's what I 
want to do— not just write papers and 
crank out graduate students. 

1 started the HIT Lab in September 
1989— one person, no furniture, no 
equipment, and 5250,000 in the bank 
from the state of Washington. Now, 
mind you, I left a laboratory that was 
the best in the world. I had 60 people 
working for me and a budget of around 
$12 to 513 million a year, not counting 
the people. I basically wiped the slate 
clean and started all over again. 



CLM: How is the HIT Lab funded? 

TAF: We basically have four sources of 
funding. The state of Washington 
provides seed funding through the 
agency of the Washington Technology 
Center. The Department of Trade and 
Economic Development actually 
provides the funds to the Washington 
Technology Center. Another source of 
funding is the grants and contracts, 
where we go out and write proposals 
for doing this work, primarily with 
industry. The third area is gifts that 
come through foundations or com- 
panies, which is the best funding 
source because there are no strings 
attached, and we don't pay university 
overhead on that money. The last area 
is through the Virtual Worlds Con- 
sortium, which now has 19 members 
(see Figure). Each of these 19 mem- 
bers, as subscribers, provides 550,000 
a year or the equivalent to join the 
Consortium. Some provide equipment 
and some provide services in lieu 
of the cash. 

CLM: There may be companies in the 
online industry that are interested in 
becoming Consortium members. 
What benefits do Consortium 
members have? 

TAF: Consortium members have 
front-row access to the research and 



Our main objective is 
to empower humans 
through building better 
interfaces between 
them and computing 
machines. 



CLM: Was it this interest from the 
private sector that made you decide 
to move out of he military 
applications? 

TAF: Yes. That's when I started 
looking around and realizing we were 
on to something really wonderful in 
terms of better interfaces between 
humans and computers. So I started 
formulating a plan to start a national 



CLM: What are the objectives of the 
HIT Lab? 

TAF: Our main objective is to empower 
humans through building better 
interfaces between them and computing 
machines. We focus on five areas to 
enable this empowerment to take place: 

1) accelerating learning 

2) enhancing creativity 

3) extending our ability to commu- 
nicate with each other from a multi- 
sensory standpoint 

4) providing a means for rapid 
information assimilation in jobs that 
need it, e.g., air traffic control and 
flying airplanes 

5) recapturing "lost world" citizens — 
ones who are unable to interact 
because of physical disabilities or 
cognitive disabilities or illiteracy. 
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A demonstration of the virtual reality hardware. 



TAF: Most definitely. We 
have two Consortium 
meetings a year, at 
which time we get into 
the technical details of 
the work. So far we've 
had three meetings. 
They all have been 
highly successful and a 
great endorsement of 
the quality of our work 
and the appropriateness " 
of our research agenda. 

One of the main 
benefits that comes from 
Consortium membership 
is not only watching this 
technology evolve, but 
interacting with other 
companies, all of whom 
are interested in birthing 
a new industry. They are 
forming their own 
liaisons with each other 
— it becomes a forum 
for discussing the evolu- 
tion of a new industry. 
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development of virtual worlds 
technology. They enjoy a competitive 
advantage as they transfer VR 
applications into the commercial 
environment Membership becomes a 
window for them to see what's 
happening and to decide what their 
own strategic plan should be. It is 
really an information source for these 
companies. There is no exchange of 
intellectual property. 

The Consortium is also a gateway 
for these companies to have contracts 
with us to work on specific a/eas. We 
make the Consortium membership a 
prerequisite before we do any contract 
work. The Consortium funds are used 
to build our generic capability, our 
laboratory infrastructure, and things 
like that. The specific contract funds 
are usually used to develop the 
applications of the technology for a 
particular customer. 

CLM: Then your relationships with 
individual Consortium members are 
proprietary? 

TAF: Yes. 

CLM: Do members have oppor- 
tunities to share information as a 
group? 
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CLM: So companies may form alliances 
with each other? 

TAF: Yes. And in a way, we act as 
matchmakers, bringing companies 
together. 

CLM: I noticed in your newsletter 
that Fujitsu Institute is a member. Are 
there other members from different 
parts of the world? 

TAF: Sharp is also a member and we 
hope to see European companies 
joining as well. 

CLM: A little earlier you mentioned 
some of the applications of this tech- 
nology. Most of the examples I've read 
about— designing buildings, modeling 
molecular interactions, teaching 
surgeons how to operate, even helping 
people improve their golf swing, 
which I might be interested in— have 
something to do with manipulating 
physical reality. 

Can you give a few examples of how 
businesses are currently using this 
technology including an example in 
which manipulating physical reality 
may not be the primary objective? 
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TAF: Well, there are three types of 
virtual worlds or virtual realities. 
There's the virtual reality where you 
are overlaying information over the 
real world. Let's say that you have a 
physical reality and you're super- 
imposing virtual images over the top 
of it to help you perform some task in 
the real world. An example of that 
would be if you were driving around 
in your car and you looked out at the 
highway and could see your speed 
projected. This is done with headup 
displays now, where you're projecting 
on the- windscreen. That is a virtual 
image, at optical infinity or some 
distance, where vou are presenting 
alphanumeric info** nation over the top 
of your space where you're driving, 
the outside world. 

Another example is a project we're 
doing with Boeing. The intent is to 
provide * superimposition of virtual 
wiring onto a form board for pro- 
duction line workers who make wiring 
harnesses that go into airplanes. They 
just follow where a virtual wire is 
located, stringing out the real wire 
over this virtual wire. So that's one 
type of virtual reality where you have 
a superimposition of a virtual image 
over a real world image. 

The second category of virtual 
reality is where you are trying to 
represent a real world scene virtually. 
That's whe^e you want to either take a 
scan of the real world scene or build a 
model of it, for example of the room 
we're in now, and then represent it in 
a virtual world so that you can 
change it Let's say that you want to 
shift the chairs around, or change the 
color scheme or the lighting. You can 
do that without having to do it 
physically. That type of virtual reality 
is using a real world source to begin 
with, then modeling it so you can 
interact with the model rather than 
have to work with the real world 
source. That is a representation of the 
world in some way either by scanning 
the real world or building a model of 
that world. 

The third type of virtual realitv is 
completely abstract. It's where there 
are no analogs, necessarily; where you 
have fields of numbers, for example, 
as you would in a spreadsheet. Or you 
have a computational fluid dynamics 
model, where the computer is just 
cranking out numbers based upon 



VIRTUAL REALITY 
GLOSSARY 

by Linda Jackson 

—excerpted with permission 
from the Vlrtuni Rodity Special 
Ripart, published by AI Expert 
; r, 415/905-2200 Vf ; « 
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artificial na^]£p^- : • >V 
Coined by art*~schoIar Myron 
Krueger' In^tli; mid-1970s 
to describe lii* Computer- 
controlled responsive 'environ- 
ments,*" which took an aesthetic 

approach ^JS!^^ 1 .*- human/ 
computer Jnter^c«3An artificial - 
reaUtyper^veT^ 
action "In terms of the body's 
rdationshifTto k graphic world 
and ^Wrat^reajponses that 
maintain the fflustan that his or 
her acHon^rerB^ing place 
within that woridJ^p%^;^ 

— M. Krueger, AriftcM Rtdlity 
II, Reading, Matsis Addison- 
Wesley; 




bandwidth ^gfftj«fa*%r 
Measurement .^(t ypically in 
Hem7 oTcydesj?er second) that 
indicates" a _ system's trans- 
mission capa ri ty7 The wider the 
bandwidthr thi^ greater the 
amount of infbrnuticrnihat can 
be transmitted inrcal time. 

^beispact^^^^^^^jrr. 
Coined by science fiction author 
William Gibson to describe a 
shared' virtual*, environment 
whose inhabitants," objects, and 
spaces arc comprised of data that 
is visualized, hearilind touched. 
It is "a consensual 7 haUucination 
experienced daily bjr billions of 
legitimate operators in every 
nation, by children taught math- 
ematical concepts. M: .a* graphic 



equations. How do you look at the 
results of that? Well, this is where you 
have to build an abstract world that 
somehow represents this information. 
There are various ways to do that. In 
the case of a computational fluid 
dynamics situation, you would create 
a flow that mimics the way the air 
would really be flowing over an 
object, let's say the wing of an 
airplane. You put the wing there and 
you visualize the air flowing over it 
This is again using the computational 
model but playing the data back in 
real time and providing a sort of 
abstract visualization of that 

Another area is where there aren't 
necessarily any correlations between 
the data and any kind of real world 
object One example would be if you 
were trying to determine if a person 
could qualify for a loan for a house, 
and you have all these disparate 
numbers and you're trying to make a 
judgment. You can build a virtual 
world that represents this inform- 
ation in a unique way, instead of 
looking at columns of numbers and 
trying *o sort ouf in your head or 
through some kind of algorithm 
whether or not this person should be 
awarded a loan or credit. Lef s say 
you have 26 different parameters. 
How in the world would you do that 
without a very complex formula? 

Well, one of the ideas that has been 
suggested is that you build a virtual 
image or a virtus I world that shows a 
person trying to go through a cave. 
The amount of debt of this person 
would change the size of the person; 
the greater the debt, the larger the 
person. The amount of the loan 
would change the size of the cave; the 
greater the loan, the narrower the 
cave. You might also have other 
protuberances, rocks that stick out in 
the pathway, depending on other 
things this person has to do. For 
example, he may have a child getting 
ready to go to college, or he may 
have a car that he just bought and 
have other payments to make on that, 
or other cash flow issues. Rather than 
looking at all these separate numbers 
and columns of numbers, you just see 
if this person can fit through the 
cave! It becomes really intuitive. And 
in a way the timing is a part of this, 
because the cave is three-dimensional 
and these protuberances may occur at 



different times, so they may be able 
to squeak by some of them. But if 
there are several occurring at once, 
they can block the passage and the 
cave that this person has to go 
through is too small for the size of the 
person at that time. So those are ways 
to take abstract information and 
represent it using metaphors that we 
can relate to. 



Rather than looking at 
all these separate 
numbers and columns 
of numbers, you just 
see if this person can fit 
through the cave! 



CLM: It seems like this application of 
representing abstract and complex 
information in three dimensions 
might be an area where companies in 
the online industry could be players. 
While i was researching virtual 
reality, I was trying to think how this 
technology could be applied to the 
kinds of data the companies in the 
online industry own. One example 1 
thought of was the information in 
medical databases. Take chemo- 
therapies for breast cancer as an 
example. There are all these different 
drugs, and different doses, and 
different frequencies, and nobody 
really knows what works. 



Could something like that be 
represented in a three-dimensional 
way to try to ascertain what the most 
effective chemotherapy is? 

TAF: res. Let me mention two areas. 
The first is called ''infomatics." There 
is a discipline now called medical 
infomatics. This is how you can take 
these complex statistical analyses, or 
even real-time data coming from a 
number of transducers monitoring a 
patient's status, and provide a rep- 
resentation of the state of that patient 
or the state of that data held. Humans 
are marvelous pattern recognizers. You 
can change those numbers into certain 
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patterns, and then look quickly for 
changes of patterns just by searching a 
visual field if you represent these 
numbers as little icons. 

For example, if you have a field of 
X's and you're looking for one O, or 
several O's and patterns of those O's, 
you can detect that quickly, whereas if 
you're just trying to look at numbers 
within a particular range, you'll never 
catch it. That is a very simple repre- 
sentation. But medical infomatics, or 
infomatics in general takes capabilities 
that we have as humans, especially our 
pattern-recognizing abilities, and 
represents information so that we can 
see patterns and see the changes in 
those patterns. 

The other area has to do with 
"knowbots." I believe ultimately that 
this is the way that humans are going 
to relate to very complex databases. 
Rather than searching databases with 
keywords, the function of a knowbot, 
especially a virtual knowbot, would be 
as an assistant, an intelligent assistant, 
an electronic associate, that you can 
send off to do things for you. Let* s say 
that you're working on a particular 
task, and you're interacting with your 
virtual world, building a building or 
something like that, and you want to 
know the soundness of building at a 
particular site, based upon the 
geological formations and the history 
of earthquakes in the area. Well, what 
you'll do is take your knowbot— 
you're an architect and it would take 
you a long time to search through 
those databases yourself . . . 

CLM: (Laughing) I know. 

TAP. (Laughing) You know. Yes. 

CLM: Much less read what you've 
gotten out of themi 

TAF: That's right. Now you take your 
knowbot. The knowbot is a virtual 
object, but you actually pick it up and 
give it a command, a verbal command. 
You say, "Knowbot, I'm going to build 
a building of this size at this particular 
location, and I'm interested in know- 
ing its history and the likelihood that 
we are going to have disturbances in 
the foundation of this building over 
the next twenty-five years. Would you 
go find that out for me?" You're 
making a verbal input to the knowbot. 



CLM: In natural language? 

TAF: In natural language. Now this 
knowbot already knows you. He 
knows your personality, your aca- 
demic background, the kind of work 
that you do, the kind of information 
thaf s going to be necessary for you to 
meet the code, and so forth. So you 
pick up this knowbot and you throw 
it into a virtual portal. This is just like 
a star gate. The knowbot goes out on 
its own searching through all these 
databases with its mission in mind, 
and also with some intelligence in 
terms of how to represent this 
information to you. 



representation of data abstracted 
front the banks of every com- 
outer in the Luznan system."" 

W. Gibson, NeuTomancer,' New 

• York, NY: Ace Books, 1984. 

^ - - — • - •■ — 




So you pick up this 
knowbot and you 
throw it into a virtual 
portal... .[and it] goes 
out on its own 
searching through all 
these databases. 



You wait for a while and go off and 
do your other tasks, and this 
knowbot comes back. It may take all 
of three milliseconds (chuckle). No, it 
might take longer than that. Bu* 
anyhow, it's gone for awhile. It 
comes back. What you have waiting 
for you is . tting there glowing. It's 
full of information waiting for you to 
access it. You put this object aside 
while you finish what you're doing. 
Then you say, okay, I'm finished with 
that, and you take the knowbot and 
you put it on, virtually put it on. So 
you pick this thing up and now you 
go inside of it. 

What it has done is create an 
alternative world for you. This world 
lets you take a tour of the under- 
ground structure of this particular area 
as a geologist understands it. You are 
able to see like you're walking through 
a museum, and you see the history of 
the seismic disturbances in the area. 
You can correlate that with sunspct 
activity, with the time of the month 
that it occurred, with the cycles of the 
earth. This is all shown to you in a 
graphic form. You can also have your 
intelligent assistant there, who visits 
with vou and has been created bv the 




DataGknrc, VR glove 
Gesture T^cognition^d^ 
e^blei juvigation^through a 
virtual' environment and inter- 
action* With 3-D objects within it 
A Bghtwieigfrt ^ove^edjwdth 
sensors that detect motion and 
send signals to the computer, the 
VR glove has fiberoptic cables 
along fingers and wrist and an 
orientation sensor jfor navigation 
and virtual bbject : navfgation« 
"DataGlove" is a trademark of 
VPL Research. ?*3EMte-M 

enabling technology 7 ^ 2 
Technology that makes "another 
technology ^s&le^TChe' vac- 
uum tube was - the enabling 
technology for both radio and 
television, ^yt^rreality 
based o^L^om^tenand head- 
^mounteif dispIajrTTias j>een 
dreaxneci^of for dfea£», Jnit Had 
to watt ^^e^^^mo^o^cs 
. of electronic mt^turfe&on, 
ucomtrot^ 
pater graphics to mature in the 
late 1980s." .^4S£ftS^ 
-^HL Rheingold, Virtual Reality, 
New York, MY: Summit Books, 



1991. 



gesture^^^ ^ 

A movement that conveys sym- 
bolic information. Usually refers to 
hand movements, but may not 
always require hands; for example, 
severely handicapped individuals 
may use foot or eye gestures to 
convey syrobolkinfonrution. 



HMD; head-mounted display 
A "helmet" with goggles contain 
ing a tiny video monitor in front 
of each eye so the image is seen 
by the wearer as three-dimen- 
sional. Often coupled with 
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earphones to cany 3-D sound, 
the HMD is the most popular 
device for placing people in 
virtual worlds. 



input devices 

Tools that transfer, data into a 
form recognizable to the com* 
puter; common input devices 
include keyboards, position- 
trackers, voice recognition 
systems, joysticks, and VR 
gloves. 



position tracker . Z. ... 
A system that tracks the move- 
ments of parts of the body and 
sends information about position 
and orientation to the computer 
for processing. Typically attached 
to the head-mounted display, but 
other hardware sensors such as 
VR gloves, joysticks/ and yolks' 
also provide digital signals 
generated by . the physical . 
actions of the participant in the 
virtual world. SW^*£ a - " ; « " : 

• .;■ : ^£sx^rt? - - ° . 
-■• ■■. ' -t^s?^/*" •' : " T .- < 
stereoscopic glasses hf- ■■. . 
Wireless,.^ b&ttirj-optTited, . 
stereo- viewingf glasses used for" 
'viewing in stereo" to look at 3- 
D computer-generated graphics.' 
Most glasses are ^infrared- 
controlled from an emitter on. a 
stereo-ready display monitor. ' 

- :■• ■. ' - J*;** ■ 

tactile feedback/force 
feedback devices- \y. 
Output devices' that transmit 
pressure, force, or vibration to 
provide the VR participant with 
the sense of touchy Tactile 
feedback simulates sensation 
applied to ' the skin. Force 
feedback simulates weight or 
resistance to motion, ; 7 '; 



telepresence. S T\' hr^ . 
The experience of immersion in a 
remote (or simulated) environ- 
ment; or, the remote operation 
systems that translate human 



knowbot. This is a person who comes 
and talks to you and tells you about 
these things as you are going through 
this museum. 

CUM: A virtual person? 

TAF: Yes. This is a virtual person. You 
can even select the personality of this 
person. This can be a really laid-back 
person that just sort of monitors what 
you're doing, and says,"Well, you 
really should look over here and see 
what this has to say." Or it can be a 
compulsive person that's really on 
your back all the time to make sure 
that you're getting all this stuff 
assimilated. 

CLM: In my opinion, the major 
problem with information at this 
stage in history is that there is too 
much. After searching all these 
databases, will the knowbot filter 
the information so I don't get 
the same thing or almost the same 
thing twice? 



Well, I believe that the 
expert searcher will 
continue to exist, but 
maybe in the form of a 
machine. 



TAF: Yes. The knowbot is your filter. 

CLM: I'm wondering if the databases 
that the knowbots access will be 
somewhat like the online databases 
that exist now, or will they be quite 
different? Will they be multimedia? 

TAF: I believe that they will be 
multimedia. We will have visual 
databases from imaging platforms like 
the LAMSAT, various NASA data- 
bases, acoustic databases, film 
libraries, encyclopedic databases, all of 
the above. 

CLM: So some text databases still, 
perhaps, but also multimedia? 

TAF: Yes. 



CLM: Today online databases fre- . 
quently require expert searchers to 
search them, either because they are so 
complicated or so expensive to search. 
Is the knowbot going to replace the 
expert searcher or will there be a need 
for some new kind of expert navigator 
in a virtual world? 

TAF: Well, I believe that the expert 
searcher will continue to exist, but 
maybe in the form of a machine. 
Basically what you want to do is 
incorporate, as much as possible, the 
attributes of that expert searcher into 
an intelligent entity that goes and 
searches these databases. On the other 
hand, there is an element of associative 
thinking of humans or inductive logic 
that is very difficult to get from a 
machine. A machine may be good at 
deductive logic, but not good at 
inductive logic, making cognitive 
leaps, getting feelings about things, 
and so forth. So I don't ever pretend to 
think that machines will truly replace 
an expert, but a machine will be able to 
accomplish 90% of what an expert can 
do, much faster. It's that extra 10% 
thafs probably going to take a human 
in the loop and if s probably going to 
be irreplaceable. However, if you can 
get your job done with 90% of it, that 
may be good enough. 

CLM: Will the knowbot be gathering 
text data and turning it into three 
dimensions? I'm still trying to see 
what place our old text kinds of 
databases will have in virtual reality. 

TAF: It will. You are going to have 
your virtual filing cabinet and your 
virtual library where your knowbot 
has actually packaged and extracted 
pieces of information, if you wish, you 
car. pull your virtual book off the wall 
or open your virtual filing cabinet. 
When you open your virtual filing 
cabinet you have all the topics the 
knowbot has accumulated for you. 
You can pull out a file folder and put it 
on your desk, and read the text. And 
you have this museum tcur as an 
initial exposure to the things that are 
going to be important. Then you can 
go to your filing cabinet and pick out 
the specifics. After that, if you want to. 
you can jack-in to the particula. 
database it came from, and go get 
further information. 
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A goggled man immersed in a virtual worid. 



The important thing to remember is 
the way we store information. Have you 
ever had the experience of reading a 
book, and you remember where a 
particular statement was on a page, but 
you don't remember the statement? 

CLM: I have to think about that I 
know what you are getting at I think I 
usually remember the words, but not 
where they are. 

TAF: Okay, well that's interesting 
because there are two different kinds 
of people. You have people who are 
spatially-oriented and others that are 
verbally-oriented. Left brain, right 
brain kind of thing. I'm a right brain 
kind of person. I can remember where 
it was, but I can't remember what it 
was, specifically. 

CLM: Well, this gets to some really 
interesting questions. When I first 
heard about virtual reality somebody 
described the glove and goggles very 
briefly to me, and mentioned 
moving through a virtual world by 
pointing with the glove. I said, "Oh, 
you know what's wrong with that? 
It's suited for people who have good 
spatial orientation, but I'm someone 
who gets lost going around the 
block." I said, "I think I might get 
lost in that kind of environment" 
Will I? 

TAF: Yes! 



CLM: So is this kind of technology not 
going to be suitable for people like me 
who are not that visually*- or spatially- 
oriented? 

TAF: If you were just going to be 
walking around the block, yes. But 
the advantage, remember, of a virtual 
world is that you can see various 
perspectives at one time. Not only 
can you be trying to find your way 
around the block, you'll have a 
miniature world in front of you 
showing you where you are while 
you're going around the block. The 
beauty of it is that it gives you a way 
to optimize the interface, adapt the 
interface to your particular abilities. 

You can take a database that was 
formerly just a column of numbers or 
descriptors, a card catalog in a 
library, or something like that, and 
you can represent it as icons, as 
windows, as doorways into other 
rooms. You'll be able to, many people 
at least will be able to, access that 
much more quickly by remembering 
the spaces where the cluster of 
information is located, rather than 
trying to do it the old way with the 
card catalog. 

Now there is another important 
thing that has to do with databases. 
How do you find your way from one 
to the other and then retrace your 
steps? This is true especially with 
hypertext. You can get lost forever in 
hypertext. 



movements into the control of 
machinery. 



virtual (-reality, -community, - 
space, -world, -environment) 

According to Webster's, "being 
such practically or in effect, 
although not in actual fact or 
name a virtual impossibility.'' IBM 
started using the word in the 
late 1960s to refer to any non- 
physical link betwe n processes 
or machines, such as "virtual 
memory," random-access memory 
being simulated using disk drives. 



VR; virtual reality 
According to VR theorists William 
Bricken and Brenda Laurel, VR is, 
respectively, "an electronically 
mediated experience" ind "a 
multisensory representation." As 
generally agreed, virtual reality 
refers to having the ability to 
interact with data in a way that 
provides the ability to "enter" and 
navigate through a computer- 
generated 3-D "world" or envi- 
ronment and change your view- 
point and interact with objects 
within that environment Virtual 
reality is characterized by 
Immersion" (it feds as if you are 
"inside" it). Virtual reality is 
interactive and the participant 
enjoys autonomy, or freedom to 
move around and manipulate 
virtual objects at will. 

Today, in virtual reality, you can 
see and hear, point and move, pick 
up things and throw them, and 
sometimes touch and feel. The 
popular concept of VR involves 
interaction with a computt*- 
generated head-rr^mted goggles, 
body suits, VR gloves and 
joysticks. VR and cyberspace are 
not the same time. 



visualization 

Formation of an image of 
(something not present to the 
sight; an abstraction); envision; 
graphical representation of data 
that normally appears in the 
form of text and numbers. 
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K'fe»B be, l if i t: 



CLM: Right! Is there some way to 
help the user so he doesn't become so 
disoriented? 

TAF: Yes. What you do is you create 
virtual breadcrumbs that you drop 
along your way. These may be sonic 
breadcrumbs. In other words, they are 
little sound beacons that you trail 
behind you as you go, which are 
associated with a spatial represen- 
tation of the database. You take this 
very complex database, and you trace 
a pathway down so you can follow it 
and come back out again, or go in 
other directions and come back to 
various points. It's very intuitive, easy 
for you to do, rather than just search- 
ing through text. It allows you to 
build mental models more easily. 
That's what virtual reality does. It 
gives you a chance to build more 
effective models in your head that 
allow you to interact rapidly, in a 
higher bandwidth. 

CLM: Can you explain how three- 
dimensional models and visual 
perception help people assimilate 
information faster? 



TAF: Well, again it goes back to, how 
do you build mental models? Because 
that's what humans do. We build 
mental models from the things that 
we see, and we extract from that 
visual experience an understanding 
or at least a representation of the 
world that we ar* experiencing. 

We also build models from ab- 
stractions, from highly symbolic 
environments. For example, with 
reading we take written symbols and 
translate them into models. We create 
worlds in our head when we read. 
Reading in a way is a guided fantasy. 
The world that we are creating is in 
our imagination, but we remember it. 
So in a way, what we are always 
doing is building models in our head 
of these places or realities. 

The real issue about learning and 
understanding is how much input 
does it require for us to be able to 
build models that are effective in 
helping us. When we learn we are 
creating better ways of interacting 
with the world so it does not require 
as much energy to do something as it 
might with an unlearned way. Or we 
accomplish something we would 



never be able to accomplish other- 
wise, like fly a jet airplane. So how in 
the world are we going to do that? 

What we're doing is providing a 
means, via building these models, of 
going back and refining those models 
and doing these particular tasks. Now 
the question becomes, if learning is 
bu ildin g models and interacting with 
those models to perform particular 
tasks, how can you accelerate learn- 
ing? Well, we believe that learning is 
accelerated when you have a way to 
provide high semantic content in 
the learning process. That usually 
means an experiential, multisensory 
input where you remove as much 
abstraction as you can. 

For example, if you are trying to 
learn about the harmonic motion of 
bodies in physics, you can read a book 
and take those words and translate 
those words into a model in your head. 
Then you can observe that model in 
your head working and come up with 
an understanding of what harmonic 
motion means in physics. Alternatively, 
if you go into either the real world or a 
virtual world— because there are many 
thiii<p you can't do in the real world 
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(for example, you can't necessarily see 
electrons, but in a virtual world you can 
represent electrons) — you are not only 
able to observe the motion of these 
objects represented in the virtual world, 
but you are able to interact with them, 
and change them and see the effect. 

You would never be able to do that 
otherwise, except in your own mind by 
looking at the equations and the words, 
and guessing about what is really going 
on. This ability can substantially 
accelerate your understanding of a 
particular concept. This is what we call 
high semantic content, high 
bandwidth— it's high bandwidth to the 
brain. You are really getting stuff in and 
out of the brain, because it's more 
tailored to the way we understand and 
interact with the world. So one of the 



values of virtual reality is accelerated 
learning— it makes it much more 
efficient to get information in and out of 
the brain. 

CLM: Are you aware of any com- 
panies in the online information 



What you do is you 
create virtual 
breadcrumbs . . .sonic 
breadcrumbs. . .which 
are associated with a 
spatial representation of 
the database. 



industry that are researching or 
developing a virtual interface to access 
commercial databases? 

TAF: Not really. Thire have been a lot 
of people who have speculated about 
it, but I believe the feeling in the 
community is that it is still a way-out, 
long-term kind of thing, rather than 
being a short-term, here-and-now 
technology. 

CLM: Can you make any predictions 
about what the status of virtual worlds 
and virtual interface technologies will 
be in, say, five years or ten years? 

TAF: Weil, I believe we will see a very 
interesting pattern emerge. Now the 
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One of Dr. Fumes*' goals for the HTT Lab is to make 
research information on Virtual reality as widely 
accessible as possible. While visaing the HIT Lab, I had 
an opportunity to "talk to To ni Emersonrwhb is the \ 
coordinator of the HTT Lab's Knowledge Base Pro jieci. . Ta 

Toni became involved With' the HIT Lab quite by 
chance. In the summer af.1991 she was attending the 
Graduate School of Libr^^ia^ uifcainaticm Science at ^ 
the University of Washington. She was working as a 
research assistant at the Odegaard Undergraduate 
library, when a ycnmg man from the HTT Lab came to the 
reference desk and asked for a bookon how-to organize a" 
library. Toni volunteered to help^ not having any idea 
what she was getting* into. As it. turns citit, her 
background in audic^video'production^Tier interest in 
multimedia databases, and her experience as a library 
research assistant were ajperfeci fit for the HTT Lab, and 
she's been working there ever since. /i^Z^.^Z^^ ■ 

A VIRTUAL UBRj^^^I^^ "4L£^ ""^rV 

One of Toni's primaryjoles in the Knowledge Base 
Project is to act as" an information gatherer and 
disseminator for the HIT lab. In addition to developing 
a hardcopy collectioii7she.is building a library in 
cyberspace. Members offfie HTT Lab fondly refer to her 
as the Information goddess* and * cybrarian.'* _, - 

When Toni joined the Lab she knew about database 
searching on services such as DIALOG, BRS, and 
NEXIS, but she credits Dan Pirone, a pr o gra mmer at the 
Lab, for teaching her how to navigate the Internet Toni 
says of the Internet, This is what the hackers like the 
best!" Toni scans print, online, Internet, and Usenet 
sources. She posts electronic information to one of her 
three internal bulletin boards: hitlJcnawledge, hitl.grants, 
and hitLnsf. 



Another part of the Knowledge Base Project is the 
sa.mrtual-XDorlds Usenet newsgroup. This is moderated 
by a HIT Lab employee, currently Mark DeLoura. (A 
/Moderator* reviews incoming messages before sending 
diem out to provide quality controL) The purpose of the 
sci.xnriual-worlds newsgroup is to transfer advanced 
.interface technology to industry. Frequent topics include 
technical discussions of vhfrial interface technology and 
its applications, virtual worlds, human-computer 
interaction, and human perception. > % ...*.: ±1 

ENIMJSERS ATTHE HTTLAB 

End-user education is another important part of Toni's 
work at the Lab. She's developed lectures which she 
refers to as LTTL HTTLs (Lunch time Interactive Technical 
Lecture at the Human Interface Technology Laboratory) 
to teach the Lab members about online and CD-ROM 
sources and how to search them. . >V--.V- •■• 7 J, 

Long-range plans include building a HTTL knowledge 
base that people outside the Lab can access. This might be 
a MUD (multiuser dimension) whereby users gain access 
to the knowledge base, select an icon, and have some 
type of multimedia experience. In addition, Toni feels 
strongly that this interaction must be a unique and 
significantly enhanced alternative to what is already 
available on the Internet * : 

Funding is Toni's primary obstacle at this point. With 
several ideas for projects that companies in the online 
industry might be interested in supporting— such as end- 
user research, multimedia knowledge bases, and living 
documentation, Toni thinks funds could be generated 
outside the Virtual Worlds Consortium for the 
Knowledge Base Project. Feel free to contact her at 
diderot@hitl.washington.edu or 206/543-5075 if you 
would like to discuss these projects or other ideas. — CM 



ERLC 



26 ONLINE November 1992 



32 



technology is immature, but it is 
growing at a fairly high rate. 1 think 
that probably by 1995 the technology 
will have gone in two directions. One 
is the low end, and the low end is 
going to be games We're going to see 
the next generation Nintendo, or the 
next couple generations of Nintendo, 
have this kind of technology in it at a 
low end. We have already seen this 
with the PowerGlove, even though 
the PowerGlove was really before its 
time. There wasn't any software to 
really use it. Furthermore, people 
found that holding up their hand all 
the time when trying to interact with 
the screen was very tiring. Then 
there's going to be the high end. This 
is where we're going to have the very 
expensive machines used in medicine, 
design, and things like that 

1 think there's going to be 
something else that happens around 
this time. Broadband telecommuni- 
cations will come with the wiring of 
the country with fiber. You will have 
a broadband telecommunications link 
that will become a ubiquitous port 
for information flow in and out of the 
home. This will support the develop- 
ment of a technology we call the 
virtuphone. The virtuphone is a 
telephone that you wear. It takes the 
place of your television and your 
telephone, and you do everything 
with it 

CLM: You wear it? 

TAF: You wear it. If s a telephone that 
you wear. 



CLM: Always accessible! 

TAF: Yes. Basically, it's the trans- 
portation system for your senses. 
Now instead of dialing a numoer and 
coupling acoustically with another 
person (with a voice phone call), or 
typing on a keyboard and having a 
screen come up, you dial a place and 
your senses go to this place. It may 
be a place where you are going to 
learn. It may be a place where you 
are se rching databases. It may be a 
place vvhere you are going to school 
or going to work. Or it may be a 
place where you are entertained. You 
may be standing on the set of King 
Lear, and you can be anywhere you 
want on the set. As a matter of fact, 



you can be inside King Lear if you 
want to be. 

CLM: Are you suggesting that people 
will no longer drive to work or to 
the theater? 

TAF: I think that there are going to be 
some changes. I don't think that thif 
will take the place of some group 
experiences, for example, the Folklife 
Festival at the Seattle Center last 
weekend. It would be tough to do 
that virtually. That was a marvelous 
celebration, and it would be difficult 
to get that ambiance virtually. On the 
other hand, again, you can do about 
90% of it virtually and get the same 
information content If s that last 10% 
that is where humans really make 
the difference. 

CLM: Will the virtuphone have an 
effect similar to that of the VCR on 
the movie business? Will people 
have the choice of staying at home 
or going to the theater? 

TAF: Yes. So why do they go to the 
movie when they can watch it 
at home? 

CLM: Because they like that big 
screen. 

TAF: That's right. Well, you'll have 
the big screen with VR. I mean, you'll 



have the biggest screen, because 
you're in it, you are in the picture! 

CLM: Finally, Dr. Furness, is there 
anything that we haven't covered that 
you would like to share with the 
participants £n the online information 
industry? 

TAF: Well, I guess to look forward to 
virtual interface technologies changing 
their business, because it's coming! 
The reason if s coming is that we can't 
hold it back, because that's the way 
humans are wired. The closer we can 
get to using fully the capabilities that 
we have, our innate capability in terms 
of our three-dimensional architecture, 
then the more effective these interfaces 
are going to be. So we are actually 
being driven by human capabilities, 
rather than by technology. 
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STR AP ON THE GEAR, FUF A SWITCH— AND V0M4, VIRTUAL REALITY. FOR TNt COMPUTER INDUSTRY, ITS A UMVtRSE Of OPPORTUNITY 



96 VIRTUAL REALITY 

Welcome to cyberspace. It's a place 
filled with computer-generated 
worlds — worlds you touch and hear 
as well as see. The possibilities are 
nearly limitless: Improved product 
design, medical breakthroughs, new 
forms of entertainment, and much 
more. There are still technological 
hurdles, but computer worlds seem 
sure to change life in the real world 
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SOUND 

Virtual Worlds can include [ 
3 D sound ^hat appears ^- 
to come from specific loca- £ 
tions. Such a system could — 
help those who monitor 
multiple sound sources 
Pilots keeping track of 
nearby traffic are an ex 
ample. NASA and Crystal 
River Engineering Inc 
Jhave developed a circuit 
board that can make 
sounds seem to originate 
from specific points and 
grow louder or fainter 



BRAIN 

Research into the human 
bra*tr>&fld behavior is shap- 
ing the design of computer- 
generated worlds. The 
goal: Present information 
so it can be absorbed and 
manipulated more easily 
and quickly. For instance, 
scientists know that the 
human mind is genetically 
programmed to pick up 
certain visual cues. This is 



helping researchers design 
better computer icons 



VISION ! 

Computer-generated 
worlds need to move with 
the speed of live action so 
that viewers perceive what" 
they see as real. Most VR is 
still too si ow, but increased 
computing power is closing 
the gap. Today's systems 
use stereoscopic displays 
with small screens encased 
in goggles or helmets. Less . 
intrusive future displays 
may resemble a pair of 
glasses. Computer- 
generated visuals could 
help with such tasks as 
repairing machines 




Adding computer-enhanced 
graphics to live video may 
make it possible to perform 
activities long distance A 
designer in Los Angeles 
might make a change in a 
simulated dashboard proto- 
type, while colleagues in 
three other cihes watch and 
see how it would look 
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VRTUAL 
REALITY 

HOW A COMPUTER-GENERATED WORLD COULD CHANGE THE REAL WORLD 



Psychologists call it "suspending 
disbelief." Computer jocks call it 
entering "virtual reality/' What- 
ever the jargon, it doesn't begin to de- 
scribe what happens in Arlington, Va., 
at the Institute for Defense Analyses. 

You sit in a wood-paneled room as 
Colonel Jack Thorpe, special assistant 
for simulation at the Pentagon's De- 
fense Advanced Research Projects 
Agency, douses the lights, flips on a 
computer — and sends three five-foot 
screens in front of you thundering into 
action. Instantly, you're transported in- 
side a tank rolling across the Iraqi des- 
ert You are performing the same ma- 
neuvers as a unit of the 2nd Armored 
Cavalry during "73 Easting," an actual 
battle in the Persian Gulf war. The 
graphics on the screens are only video- 
game quality. Yet. the illusion works: 
You duck as shells scream toward you 
and explode in ear-splitting fury. 

It isn't unusual for soldiers participat- 
ing in this exercise to curse or sweat as 
the computer-simulated fight unfolds. 
Something else happens as well: Their 
scores for battlefield acumen improve 
dramatically after they practice with 
these video tank crews. In an era of 
shrinking defense budgets, such training 
offers invaluable experience without the 
cost, damage, and logistical hassle of 
war games. "We will expect a smaller 
military to be masters of a wider ensem- 
ble of skills," says Thorpe. 'This is an 
idea whose time is right." 
niw sensations* The cyberspace tank 
battle is primitive compared with visions 
of "virtual reality" trumpeted in books, 
movies, and the TV show Star Trek: The 
Next Generation, There, intergalactic 
travelers use computers to conjure up 
Sherlock Holmes's London or a sexy 
date. But as daRPa's system proves, 
computer-generated worlds don't have to 
be super-realistic to evoke real life. 



That fact is turning virtual reality into a 
red-hot technology. 

There's plenty of confusion over what 
vr is. But to most developers, the core 
of every system is a data base that con- 
tains data from a brain scan, specifica- 
tions for a car dashboard, the descrip- 
tion of a fictional landscape— in short, 
data that can represent almost anything. 
A powerful computer with sophisticated 
graphics then renders a "world," often 
in &-D, that recreates precisely what the 
data describe. VR dis- 
plays vary widely, 
from images on a com- 
puter monitor to the- 
ater-style displays 
such as 73 Easting to 
projections on stereo- 
scopic lenses mounted 
inside helmets that VR 
participants wear. 

Whatever the ap- 
proach, two character- 
istics distinguish VR 
worlds from other 
computer graphics: In- 
creasingly, they con- 
vey multiple sensory 
information — sound or 
touch— to make envi- 
ronments more realis- 
tic. And they are 
interactive. In some 
systems, a viewer 
wearing a sensor-laden glove manipu- 
lates objects in the computer as one 
would naturally. In others, images on 
the screen or a viewer's perspective are 
manipulated with a mouse or joystick. 

At IBM's Watson Labs in Hawthorne, 
N. Y M for instance, an engineer seated in 
front of a projection screen, looking at a 
sleek, beige dashboard becomes a test 
driver for a 1997 Chrysler. Wearing 3-D 
glasses and a glove with sensors, he 
turns the steering wheel ana reaches for 





buttons as though in a real car. Chrysler 
Corp. is developing the system with IBM 
in hopes that the exercise could cut 
months off the three-year to five-year 
car-design process by letting engineers 
spot inconveniently positioned knobs and 
other problems before they surface in 
expensive prototypes. 
•PAST the hyps.' Intrigued by this kind 
of potential, dozens of government, uni- 
versity, and industrial labs, from NASA 
and the Defense Dept to the University 
of Washington <UW), 
are embracing virtual 
reality. In the next 
four years, the mili- 
tary hopes to spend 
more than $500 million 
on simulations. This 
fall, the Army will like- 
ly award an additional 
$350 million, eight-year 
contract to create an 
advanced network for 
battlefield simulations. 
Industry giants — in- 
cluding Boeing, AT&T, 
Sharp, and Fujitsu- 
are investing millions, 
too. At uw's Human 
Interface Technology 
Laboratory, some 19 
companies have cre- 
ated the Virtual 
Worlds Consortium to 
apply VR to business. "Forget the games 
and electronic sex," says Bryan Lewis, a 
researcher at IBM. "We are past the 
hype and pursuing real applications." 

This could be a boon to computer gi- 
ants such as IBM, DEC, Apple, Sun, and 
graphics workstation maker Silicon 
Graphics, vr represents a potentially big 
market— and a flashy selling point— for 
their muscle machines. Startups includ- 
ing Exos, Virtual Vision, and Fake Space 
Labs are building gear to enhance vr 



Cyberspace 
simulations may 
enhance job 
performance and 
training, improve 
product design, 
assist surgeons, and 
create interactive 
forms of entertain- 
ment, but it will be 
years, if ever, 
before all that 
is a reality 
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worlds — viewing devices, acoustical 
chips, and sensors. Autodesk, Sense8, 
vpl Research, and others see their for* 
tunes in systems that business can use. 

For good reason. Cyberspace worlds 
that exist only in the electronic ether can 
be a powerful too! in the hands of archi- 
tects, engineers, and scientists. They can 
also be used to boost productivity, im- 



prove product design, and provide more 
cost-effective training. In medicine, vr 
tools are being used to create 3-D X-rays 
to help surgeons plan procedures or as* 
sist in surgery miles away* Psycholo- 
gists want to use the technology to treat 
patients and to study human behavior. 
Artists and entertainment moguls are 
pioneering new attractions — interactive 




theater, interactive fiction, and even vir- 
tual sculpture, cyberspace works that 
defy the laws of physics. 

Whether VR systems will ever match 
the sophistication they display in fiction 
is far from certain. The field faces huge 
technical hurdles: Success will depend on 
improvements in hardware and soft- 
ware, plus new insights into the human 
brain and behavior. And as systems be- 
come more "real," they will pose thorny 
ethical questions: Could VR influence 
people in pernicious ways that conven- 
tional media cannot? 
Still VR's social and economic poten- 
tial seems clear. 



RESEARCHERS USED A 'DATAGLOVF TO ANALYZE OEM ENS' MOTION 



ANATOMY OF A FASTBALL 



Hurting a 1 O0-mile-per-nour fastball 
down the middle it a special skit! worth 
analyzing. But a big-league pitcher's 
ami, wrist, and finger movements change 
so rapidly that they're almost impossible 
to dissect. This thwarts efforts to teat?) 
from good pitchers— or figure out what's 
wrong when they have injuries or slumps. 

Insights into these puzzles of move* 
ment could come from a new data-collec- 
tion tool that's integral to virtual reality, 
Greenleaf Medical Systems, a four-year- 
old startup in Palo Aho, Calif., has li- 
censed the "datagiove" from vn Re- 
seardi Inc. for medical uses. A black 
Lycra glove with fiber-optic cables at- 
tached relays movement signals to a 
computer, which quantifies hand motion. 
In a recent experiment, Greenleaf put 
datagloves on the hands of four Boston 
Red Sox pitchers, including team ace 
Roger Gtmens. 



Attached to an Apple Macintosh com- 
puter, the glove recorded subtie rekrHon* 
ships between speed, position, flex, and 
other variables as the four men threw a ' 
variety of pitches. For every thr ee-seco nd 
prtch, the system co mpiled 16,000 data 
points. Red Sox associate team physician' 
William J. Morgan h building graphic im- 
ages to see what he con team, iy repeat- 
ing the experiment, he hopes to identify 
movement changes that make a pitcher 
less effective—- and correct them. 

Company founder Walter J. Green* 
leaf sees broader potential in the experi- 
ment: He envisions a huge manW j ana- 
lyzing repetitive-stress injuries, on 
increasingly common malady of office 
workers, and in diagnosing other ortho- 
pedic and neurological ills. He also hopes 
to make patients who can't speak able to 
communicate through gestures the com- 
puter interprets. 



Democratic Vice 
Presidential hopeful 
Al Gore considers 
vr so crucial to 
"the way we design 
new products, teach 
our children, and 
spend free time" 
that last year he 
chaired hearings on 
its value to Ameri- 
can competitive- 
ness. The conclu- 
sion: The U.S. is 
underinvesting in 

the technology. 

To vr advocates, thafs a mistake. Vir- 
tual reality represents "the manifest 
destiny for computers," asserts Eric 
Gullkhsen, founder of VR software pro- 
ducer Sense8. By creating worlds of col- 
or, shapes, sounds, and feel, these sys- 
tems should amplify the powers of the 
mind to see previously hidden relation- 
ships in complex sets of data and to 
absorb, manipulate, and interpret infor- 
mation more quickly and completely. 
The distinction between immersion in a 
VR world and analyzing the same infor- 
mation using blueprints, numbers, or 
text "is the difference between looking 
at an aquarium and putting on your scu- 
ba gear and diving in," says Thomas 
Furness, director of uw's Human Inter- 
face Technology Laboratory. 
bump AMD out. Just ask engineers at 
Northrop Corp., who are using a vr sys- 
tem from Sinographies Engineering 
Corp. to help redesign the Air Force's F- 
18 fighter jet They model air-intake 
ducts on computers to make sure they 
fit through bulkheads, rather than build- 
ing expensive hard models. An operator 
wearing wraparound goggles moves 
parts around with a type of mouse, mak- 
ing sure they fit together in virtual 
space. The software even simulates re- 
sistance, so engineers know when parts 
"bump" against each other Project En- 
gineer Robert E. Joy loves the flexibili- 
ty: "It's like reaching into the work- 
station and grabbing the part/' he says. 
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vr represents the second major effort 
in two decades to bring about a dramatic 
evolution in computers. The aim of the 
first, artificial intelligence, originally 
was to build systems that could mimic 
human reasoning, a goal that has yet to 
be reached. Virtual reality is the antithe- 
sis of what ai tried to do. It aims "to 
extend the power of the person" says 
Robert Jacobson. president of Worl- 
Design, a Seattle vr software" startup. 

That's what a visualization tool de- 
signed by Maxus Systems International 
does for managers at TIAA-CREF, a New 
York pension fund with $105 billion in 
assets. Tracking the performance of a 
group of stocks against the larger mar- 
ket is a challenge for analysts, who 
must follow hundreds of ever-changing 
numbers. Using software from Sense8, 
the Maxus system converts the numbers 
to a 3-D schematic of colored squares 
that move and symbolize individual 
stocks within grids representing market 
and industry sectors. It runs on a per- 
sonal computer and draws on real-time 
feeds from financial wires. 

A specialist in bank stocks may glance 
at the computer and notice that a box 
showing banks in the Pacific Rim is ac- 
tive. The squares are red, a signal that 
the stocks are falling. The analyst uses a 
mouse to "fly" into the lowest tier of 
stocks, which have plunged the fastest, 
and click on the security that has 
dropped most. Up pops text on that 
bank. The process takes seconds, so 
portfolio managers can "identify trends, 
recognize exceptions, and make deci- 
sions more quickly," says Sense8 Presi- 
dent Tom Couli. 'That can translate into 
a tremendous amount of money." 
nriNG mice. Such a system falls short 
for vr purists, who argue that only an 
immersive experience with a helmet 
holding two stereoscopic screens and 
headphones will do. That way, you see 
and hear only what the computer gener- 
ates, interacting with the environment 
as in the real world. At NASA Ames Re- 
search Center in Mountain View, Calif., 
this approach lets you look around the 
surface of Mars, which has been recreat- 
ed from satellite data. A motion sensor 
in the helmet lets you look in any direc- 
tion, and the computer rerenders the 
scene to reflect your new perspective on 
the Martian landscape. 

Still, theater-style simulations and 
two-dimensional computer displays can 
be just as powerful. Using a Silicon 
Graphics Inc. system, urban planners in 
Los Angeles are building an 80-block-by- 
80-block virtual model of renovation 
plans for riot-damaged areas. The value: 
It's hard for untrained ■ >ople to read 
blueprints, and models are expensive. 
Yet, community involvement is essential. 
This way, residents can use a mouse to 
"fly" through the streets as if they were 



GRISLY GRAPHICS: FROM THE V* RECREATION OF A 1991 SHOOTING 



IS VR REAL EHOUGH FOR THE COURTROOM? 



On the night of Feb. 27, 1991, Son Fran- 
cisco porn-movie king Jim Mitchell drove 
to the home of his younger brother and 
business partner, Artie, in Corte Madera, 
Calif. Minutes later, Artie was dead, and 
a dozed Jim was arrested walking away 
from the scene. 

The district attorney didn't buy Jim's 
story that he shot eight times in self-de- 
fense, frightened by a beer bottle an in- 
toxicated Artie wielded in a dim hallway. 
But there were no eyewitnesses. So, the 
prosecution persuaded the judge to let 
the jury watch a video of Artie's death. 

This was no candid videotape, howev- 
er. In the first-ever use of vt in a criminoi 
trial, a ballistics expert recreoted the 
event, complete with bullet trajectories, 
on o personol computer using computer- 
oided design software from Autodesk 
Inc. In the animation, a ghostly figure 
peeks from behind o door. The figure 



emerges and walks stiffly down a hail- 
way. A red tube pierces, then exits, the 
body. The figure continues to walk until 
another red tube strikes its forehead. 

The video, which was created after 
analyzing evidence found at the scene, 
had the desired effect. Last Feb. 19, the 
jury convicted Jim Mitchell of man- 
slaughter and sentenced him to six years 
in prison. He's appeoling, in part be- 
cause of the videotape, which his attor- 
ney, Nanci Garence, colls "wizardry thot 
has no place in a court of law." 

The Mitchell case highlights the ethical 
dilemmas inherent in vr. Reality is, after 
all, more than sophisticated softwore. To 
the defense's chagrin, for instance, the 
figure in the tape doesn't wield his beer 
bottle in o remotely threatening way. In 
short, v* may raise a thorny question for 
judges: Even in the best of simulations, 
can reality be manipulated unfairly? 



in a helicopter. And designers can pop in 
a park bench or delete a 7-Eleven, test- 
ing suggestions from those who live in 
the real Los Angeles. 

The idea of using computers to render 
useful environments dates back to the 
1960s. Back then, however, the comput- 
ing power needed to generate even 
crude 3*D graphics was so expensive 
that only government agencies such as 
Defense or NASA, plus a few university 
labs, could afford it. Even today, special 
helmets used for military flight simula- 
tors can cost Si million. 

The field began to attract attention 
when onetime computer hacker Jaron 
Lanier coined the term virtual realitv in 



the mid-1980s. In 1984, he founded VPL 
Research Inc. in Foster City, Calif.— the 
first company dedicated to vr worlds 
(page 104). vpl has developed key vr 
aids — head-mounted stereo screen dis- 
plays, or "eyephones," plus the "data- 
glove" and the "datasuit," which let vr 
viewers convey information to comput- 
ers with hand signals. Don a Dataglove, 
and an image of a hand appears in the 
virtual world, so you can point to ob- 
jects, pick them up, or command the 
computer. 

More than anything else, though, the 
relentless increase in performance— and 
decrease in price— of semiconductor 
chips is driving vr by allowing computer 
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makers to build more sophisticated 
graphics systems. At the high end, Sili- 
con Graphics* new $100,000 "Reality En- 
gine" has a computing speed 1,000 times 
as fast as most PCs. allowing it to pro- 
vide quick rendering and real-time mo- 
tion in vu worlds. On the low end. desk- 
top vr systems based on Intel Corp.'s 
486 chip cost as little as $20,000. Richard 
H. Dym. general manager for multime- 
dia at Autodesk Inc.. calls new program- 
ming tools and applications for these 
systems the leading edge of software 
development. 

Entertainment is one of the first bene- 
ficiaries. Nintendo Co.'s 
$99 Powerglove, a sim- 
pler version of VPl/s 
$8,800 Dataglove, lets 
video-game wizards 
play with hand gestures 
and has already helped 
spawn a host of VR-like 
video games. Virtual 
World Entertainment 
LP's vr game site in 
Chicago, the "Battle- 
tech Center." has sold 
some 300.000 tickets at 
$7 each since it opened 
in July to players who 
sit in an enclosed cock- 
pit to engage in Star 
Wars -like battles. The 
company has two sites 
in Japan and plans to 
open 17 more over the 
next three years. 
'TtiEPRESENCE.' In busi- 
ness, much vr technol- 
ogy will evolve out of 
current computer sys- 
tems. Computer-aided 
design, or CAD. systems 



training plant operators. Today's train- 
ing rooms for fossil-fuel plants cost up 
to $1 million. Using vr, the cost might 
dip under $100,000. And eventually, says 
Hugh W. Ryan, director of new-age sys- 
tems for Arthur Andersen Consulting, 
vr worlds will be used to simulate busi- 
ness interactions — from sales negotia- 
tions to general management prob- 
lems — and will replace some of today's 
expensive seminars and classes. 

vr may also help train workers for 
flexible manufacturing. Boeing Co.'s 
project manager for human-computer in- 
teractions, Keith Butler, is developing 
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have been around for years. Adding VR's 
greater resolution and interactivity can 
enhance their utility, as Chrysler, among 
others, is discovering. 

"Telepresence," a vr tool that refers 
to the remote manipulation of equip- 
ment, shows similar potential. The Japa- 
nese construction company Fujita Corp. 
has hired VPL to help it build a system 
that lets an operator in Tokyo direct a 
spray -painting robot anywhere in the 
world. The operator views the building 
to be painted on a computer, then works 
controls that signal the robot to spray. 
With VR. the image is so painstakingly 
exact that the human operator makes no 
mistakes in directing the operation. 

In business education and job training. 
vr's chief benefit would be lower costs. 
The Electric Power Research Institute 
has teamed up with MITRE Corp. to de- 
termine if an electronic mock-up of a 
power-plant control room using stereo 
projection displays can be effective in 



VIRTUAL MODELS: Northrop Corp. 
engineer Robert Joy uses a vr system to 
design parts for the Air Force's F-18 
fighter jet in virtual space, lessening the 
need for expensive, time-consuming 
physical mock-ups 

techniques to project job instructions 
onto see-through goggles worn by as- 
sembly workers or onto the work space 
in front of them. In theory, instructions 
presented this way could replace hours 
of training in which workers learn jobs, 
then must be trained again when the 
task changes. With such displays, a 
worker might assemble wing flaps, then 
switch to nose cones on the same day 
with little loss of productivity. 

In perfecting such systems, develop- 
ers must solve some novel problems. 
Why do some people become nauseated 
when navigating j n cyberspace? And if 
you have to make a trade-off between 



complex, realistic graphics or live-action 
motion, which is more important for 
maintaining the illusion of reality? 

The answers to such questions lie in 
the cognitive and behavioral sciences. 
Greater knowledge of the structure of 
the brain, how it processes information, 
and how people think and perceive is the 
key. Such research already indicates 
why VR worlds are so effective in train- 
ing, says Roger Shank, director of the 
Institute for the Learning Sciences at 
Northwestern University. Studies show 
that in general, people reason or solve 
problems based on cases, examples, and 
experience, not by 
learning rules. "That's 
why the flight simula- 
tor is the best piece of 
educational software 
ever made/' savs 
Shank. 

OSMETIC ctlis. One of 

the key assumptions of 
VR work is that the 
brain can process infor- 
mation better when it 
is presented through 
sight, sound, and touch 
instead of just text or 
numbers. Scientists also 
are finding that the 
responses to certain 
visual cues — including 
hand-eye coordination 
and the ability to detect 
the edges of objects and 
to recognize movement 
across a meadow of 
grass — are encoded in 
genes. Our cave-dwell- 
ing forebears originally 
developed these re- 
sponses in reaction to 
the world around them, says Ronald M. 
Pickett, professor of psychology at the 
University of Massachusetts at Lowell. 

Pickett and others are designing soft- 
ware icons that mimic those cues. "We 
want to trick the visual system to evoke 
quick, natural perceptual processes in 
the sen-ice of analyzing datr." he says. 
To do that, he has created an icon that 
looks like grass. It changes length, 
curve, and arc to represent numeric data 
such as income level, age, and sex. Each 
icon can convey multiple characteristics 
that can be comprehended at a glance. 

Whether people experience virtual 
worlds as "real" doesn't depend entirely 
on real-time motion, graphics, or visual 
cues, however. One of the most difficult 
challenges is to imbue computer charac- 
ters with humanlike qualities. As part of 
that effort, Joseph Bates, a computer 
scientist at Carnegie Mellon University 
in Pittsburgh, is trying to create vr dra- 
ma — interactive programs in which com- 
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WAR GAMES: The Pentagon is 
sharpening soldiers* skills with 
computer-generated battlefield 
exercises that unfold on huge screens 

puter characters and people collaborate 
to create stories or situations. At first, 
it's hard to understand how an animated 
landscape with four bouncing blobs 
could be relevant. The blobs' only activi- 
ty is jumping up and down, and they are 
supposed to take turns "leading." But 
when one ball starts to dominate the 
activity, the others react. They change 
color, or slow down. One even turns 
from red to blue, retreating to a corner, 
its sides heaving, to . . . weli. sulk. 

The balls appear to be exhibiting emo- 
tion and acting independently because 
Bates and his colleagues have pro- 
grammed them based on theories of be- 
havior. These hold that emotion — and 
the behavior that results from it— arise 
from goals that are being met, opposed, 
or otherwise affected. When pro- 
grammed this way. the blobs begin to 
act as if they have "personalities,** and 
people can identify with them. 
'■akfogenic zone. 1 Building on such 
work, researchers one day hope to popu- 
late virtual worlds with creatures — hu- 
man-looking or not — that people interact 
with as they would anotner person. 
These characters might analyze a prob- 
lem, monitor an experiment, or play the 
role of someone in a business simula- 
tion—a hot sales prospect, say. They 
would probably react to voice commands 
but would also need to convey and un- 
derstand more subtle human communi- 
cation such as body language. Sound 



fantastic? Not to Fujitsu Ltd, which has 
invested $250,000 in Bates's work. His 
work reinforces Fujitsu's research in 
"artificial life," computer algorithms 
that behave like biological entities and 
could become the basis of computer-gen- 
erated characters in VR worlds. 

Fine-tuning the sensory and psycho- 
logical factors that make a vr world 
"real" is a further technical challenge. 
Experience shows that VR viewers adjust 
to low-resolution monitors. The brain 
also accepts slow, jer- 
ky frame speed and 
much faster live ac- 
tion — 30 frames per 
second. But in between 
lies what Thomas P, 
Piantanida, principal 
scientist of SRI Interna- 
tional's Virtual Percep- 
tion Program, calls the 
"barfogenic zone" — 
from 4 to 12 frames 
per second. At that 
speed, the confusion 
between what the 
brain expects and what 
it sees can make view- 
ers sick. Until comput- 
ers can create complex 
worlds with live mo- 
tion. Piantanida's work 
suggests that it's bet- 
ter to run crude dis- 
plays faster than to 
run detailed displays in 
the barfogenic zone. 

Putting sound to vir- 
tual worlds is one more 
key to improving peo- 
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THE BLOBS: Carnegie 
Mellon University's Joseph 
Bates has created animated 
characters that exhibit 
emotions based on theories 
of human behavior. The 
next step: Virtual worlds 
filled with creatures that can 
interact with humans 



pie's ability to absorb infor- 
mation. "Our ears point our 
eyes," says NASA Research 
Psychologist Elizabeth M. 
Wenzel, an expert in add- 
ing 3-D sound to virtual en- 
vironments. A military pi- 
lot, for instance, often 
monitors as many as eight 
conversations from air and 
ground sources through 
the same earpiece, Wenzel 
says that making the sound 
appear to come from differ- 
ent directions helps pilots 
key in on high-priority in- 
formation. A new circuit 
board developed by NASA 
and Crystal River Engi- 
neering Inc. that produces 
3-D sound will make it easi- 
er to put sound in virtual 
worlds. The chips mimic the 
shape of sound waves as 
they hit the human ear 
from different directions, 
creating the illusion of dis- 
tance as sounds grow louder and softer. 

VR researchers are opening another 
portal to the brain through so-called 
force feedback. The idea is to build 
weight, resistance, or attraction into joy- 
sticks, so* that VR voyagers can "feel" 
simulated objects. Researchers at Digi- 
tal Equipment Corp. are working with 
outside chemists to simulate the forces 
of molecular attraction and repulsion. 
Their goal is to develop a system within 
two years that will help chemists feel 
these forces as they 
experiment with 3-D 
images of molecules to 
develop drugs and oth- 
er chemicals. That's im- 
portant because mole- 
cules that appear to be 
compatible often are 
not. Knowing this 
in advance could help 
scientists avoid blind 
alleys. 

The more sophisti- 
cated vr worlds be- 
come, the more contro- 
versy they may gen- 
erate. Some psycholo- 
gists want to use VR in 
psychotherapy to alter 
the perspective of pa- 
tients, or to recreate 
environments that 
cause stress or other 
problems as a way to 
help treat phobias, de- 
pressions and schizo- 
phrenia. British psy- 
chologist Peter Ward, 
who plans to use vr to 
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o tfet a feel for what 
is different about 
virtual reality, meet 
Jaron Lanier, chairman of 
vpl Research Inc. Dread- 
locks crown his ample 
frame. His Sausalito 
(Calif.) studio — he's an ac- 
complished musician — is 
filled with exotic instru- 
ments. On the door hangs 
his image emblazoned on a 
psychedelic poster. The 
poster is hot in Europe, 
where VR is tres trendy 
and Lanier is a cult figure, 
reflecting vpl's preemi- 
nent role among the start- 
ups that are pushing the 
technology's frontiers. 

Lanier/ 33 r started VPL 
in his garage eight years 
ago with money he made 
from programming an 
Atari Corp. video game 
called Moondust. Fiddling 
with icons and graphics he 
hoped would make math 
easier led him to a more sweeping vi- 
sion. Today, vpl sells hardware devices 
such as the Dataglove and Datasuit for 
navigating in virtual space, helmets 
that surround you with computer-gen- 
erated worlds, and programming soft- 
ware that even children have used to 
create virtual environments — kids, and 
a few other customers, such as Boeing, 
SRI International, Matsushita, and MCA. 
the 'BOOM.' vr's big winners eventual- 
ly should be heavyweights such as In- 
tel, IBM. Apple, and Silicon Graphics— 
the makers of graphics chips and com- 
puters. Alan Meckler, 
publisher of the news- 
letter The Virtual Re- 
ality Report, sees lift- 
off toward the end of 
this decade. But who- 
ever cashes in will 
owe a debt to VPL and 
many other innova- 
tors. Crystal River 
Engineering in Grove- 
land, Calif., is selling 
acoustical circuit 
boards that let pro- 
grammers put 3-D 
sound — say. the sound 
of a door opening and 
closing — in a virtual 
space. Fake Space 
Libs has invented a 




stereoscopic viewing device called the 
"boom" — as in boom microphone — that 
lets a person move around a virtual 
space by looking through a viewfinder. 

Lanier thinks medicine will be VR's 
"monster market," partly because of 
the need for better visualization of di- 
' agnostic scans. At a recent San Diego 
conference, surgeon-inventors mingled 
with science fiction writers, while Sony 
Corp. marketers pitched high-defini- 
tion-television screens. The other prod- 
ucts discussed ranged from systems 
for doing remote surgery to 3-D data 



PIONEERS IN VIRTUAL REALITY 




THESE VR LEADERS ARE ALL PRIVATE COMPANIES 
Compony Location EmptoyMf 


Founded 


vfi foster city, cauf. 25 

Makes hardware and software, induding DatoGlove and EyePhones 


1984 


:rvr. sausalito, cauf. 8 

Creates programming packages for virtual worlds 


1990 


Makes stereoscopic viewing device 


1989 


: SOUTH PASADENA, CAUF. 15 

Develops engineering visualization programs 


1985 


r; "::;:.£ palo alto, calif, 12 

Adapting the DataGlove far use in median* 


1988 



W0BURN,MASS. 
Sells a device that signals computers through gestures 



25 



bases for analyzing casu 
alty data in a war. 

The core of such mar- 
kets will be software, says 
Robert Jacobson, founder 
of WorlDesign in Seattle. 
StereoCad in Sunnyvale, 
Calif., and Virtus in Cary, 
N.C., specialize in archi- 
tectural and engineering 
design programs. BioCad 
in Mountain View, Calif, 
sells "virtual chemistry" 
software that lets scien- 
tists create 3-D, interactive 
models of molecules and 
other chemical structures. 
Engineering Animation 
Inc, in Ames, Iowa, makes 
3-D graphics and anima- 
tion programs that re- 
create accident scenes for 
use in court Both Sense8 
and vpl sell "tool-kit" pro- 
grams for VR software 
programmers. But they 
may not rule the market 
for long. Autodesk Inc. in 
Sausalito, which has 700,000 customers 
for its computer-aided-design software, 
could have an edge when it comes out 
soon with its own tool-kit program. 
utter DfTtn. Such competition has 
begun to alarm Lanier, a major vpl 
shareholder. In May, he named a new 
chief executive: ex-Hewlett-Packard 
Co. executive Walt Fischer, As white 
collar as Lanier is not, Fischer may 
have better entree to corporate cus- 
tomers. This could be crucial for VPL's 
plan to become a systems integrator, 
selling packaged solutions — not just 
components. "We've 
sold millions of dol- 
lars' worth of hobby 
stuff," says Lanier. 
'The transition now is 
into a real company." 

Whether vpl and 
the other vr upstarts 
will prosper is impos- 
sible to predict. But 
even if they do not, 
pioneers such as La- 
nier are trailblazing a 
technology that is 
likely to benefit every 
industry that relies on 
computing. 

By Joan O'C. Hani 
it ton 
Calif. 



in Sausalito. 
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the tame desk— with the ability 
even to hand electronic documents 
back and forth. Some visionaries 
even predict the advent of highly re* 
akstic computerised sea. 

Fantastic stuff, and It already fa 
rafaing *ieettoue about whether vir- 
tual reality can be abused, like a 
drug, to create alternate psychedelic 
environment* in which users couU 
disappear from the real world. "Elec- 
tronic LSD," some 
cal it There even 
are fear* among 
aome in the VR 
field that the gov- 
ernment win* seek 
lo regulate vktnal 
reality. 

The problem 
with virtual reali- 
ty Is that It's s 
door to another 
world." said Mi- 
chael McGreevy. 
head of NASA* 
virtual reality re- 
search effort and 
a pioneer in the 
field. "People 
might walk 
through it and 
choose not to 
come back." 

Even in Ua 
more conserva- 
tive forma, virtual 
reality can pro- 
duce a disorient- 
ing and nauseat- 
ing feeling known 
as "simulator sick- 
ness," In which 
the eyes tell the 
brain the body is 
rooting, based on 
the iraagea from 
the VR system— 
contrsry to the 
signals front the 
inner ear. which 
thinks the body is 
stationary. 

But experts say 
the positive attri- 
butes of VR far 
outweigh any po- 
tential negatives. 
Virtual reality, 
they say. is the 
best example yet 
of how powerful 
computers will 
change We — quite 
literslly. in this 
case. 

In virtual reali- 
ty, what happens is this: We're put- 
ting the observer Inside the medi- 
um," said Thomas A. Fumess III, 
who began working on virtual reafity 
technology for the Air Force more 
than 25 years ago and now b direc- 
tor of the Hunan Interface Technol- 
ogy (HIT) Laboratory at the Univer- 
sity of Washington in Seattle, one of 
the leading virtual reality research 
facilities. "Boy, let me tell you. k 
changes the whole rules." 



In s way. vktasl reality is the ukt- 
mate optical fesfe*. Meaty, the us- 
er has the feeing of being hi an a+ 
temste world, one whose attributes 
can be airything k* designer wants 
them to be. "We can change gravity, 
we can change the speed of light, we 
can duuge the speed of sound," Fur- 
ansa said. "We are s i pewa ri at" 



w* be fee once k is fatft-and to 
decide on design changes that might 
otherwise not be abvfauu untl con- 
struction started. 

One experimental system snows 
an architect to awe through the de- 
sign of a veteran* hospital fa a virtu- 
sl atoeSdn* te teat access to doors, 
hallways, sight switches and other 
detlf ■ desert*. Aad Matuthlta 
Bectrfc b*mM Ce, Mtt Jepeww 




ERIC 



Some of these new worlds wilt be 
more than s little surrealistic. "Neo- 
romaneer" a 1 W4 novel by science 
fiction novelist WllKam Gibson that 
has been the inspiration for many in 
the virtual teality field. Imagined a 
hallucinatory world known aa "cyber- • 
space," in which usera connected' 
their brains to gigantic graphical 
databases and wandered about them 
electronically fa search of interna* 
tkm. 

That's not so farfetched. Many 
personal computer programa now 
can convert numbers from spread- 
sheets Into sophisticated three- 
dimensional charts. Virtual reality 
puts the user into the middle of 
those graphs jnd animates them. 

"You might ace ebbing and flowing 
of tbe tide (of data), oceans rising 
and bBing, trees growing, some sort 
of organic model" of data, McGreevy 
said. Applied to the analysis of finan- 
cial markets, a virtual realrty-based 
system could be s powerful tool If 
you liked program trading, you'll 
love virtual reality trading," 
McGreevy said. 

Other virtual reality examples are 
a tittle more like real fife. There al- 
ready are several products that al- 
low architects to convert computer- 
ized plans Into three-dimensional 
computer-generated models of buld* 
ings and then use virtual reality to 
"wait* through them. Unlike a blue- 
print or even a computer-generated 
design, virtual reality snows archi- 
tects to experience what s buildmg 



giant, has set up s virtual reafity ar- 
chitectural system in s Tokyo de- 
partment store to gfve customers 
lifelike previews of what rrmnsV hng 
could do for their kitchens. 



The typical virtual reality setup 
currently fa voire* four or five major 
components. One fa a helmet with a 
nwnfature television screen for each 
eye tn provide three-dimensional vi- 
suals. The second Is s Joystick or 
bai-fike control device that fa used 
to direct moveanent The third fa s 
"Pomensus™ sensor, suspended sbove 
the user, that constantly senses the 
locations of the helmet and control 
device and t ra n sm its this informa- 
tion hack In the fourth and perhaps 
most important component, a com- 
puter, which generates the sound 
and graphics piped into the helmet 
and keeps track of the user's posi- 
tion In this artJfkfaUy generated 



working on hghter, more comfort- 
able devices, most notably replace- 
ments for the boacy helmet. "Hef- 
meta are Stone Age," seek red Ken 
Pimentsl, product manager at 
Senses, a Sausalito. Calif.-based 
company that writes computer soft- 
ware used in VR systems, including 
the stock-trading system, which fa 
viewed on s regulsr television 
screen rather than from a helmet. 

Tbe goal fa to 
come up with a 
virtual reality in- 
terface that fa as 
easy to wear as s 
pair of glasses. "If 
the consumer 
market comes out 
with some effec- 
tive, lightweight 
glssaes, it could 
drive the market, 
even If It's low 
quality ," said John 
Latta. president 
of Fourth Wave 
Inc.. an Alexan- 
dria consulting 
firm active ki vir- 
tual reality work. 

The University 
of Washington's 
HIT tab fa testing 
a visual device 
known as a "virtu- 
al retina display " 
in which a filtered 
laser draws imag- 
es on the user's 
retina — s scarify- 
ing and currently 
crude technology 
that experts none- 
theless believe 
holds great prom- 
ise. Another alter- 
native does away 
with head-mount- 
ed displays entire- 
ly by ci'iwtructing 
virtust reslity 
viewing rooms 
with Urge-projec- 
tion TV screens 
as the watts, ceil- 
ings and floors. 

Abo crude, for 
the moment, sre 
the imagea pro- 
jected of the vir- 
tual world. Be- 
cauae of 
hmttationsof even 
the most powerful 
computer graph- 
ics systems, the 
world of virtual reality often looks 
like a schematic drawing populated 
by a handful of slightly more realistic 
cartoon characters; the atin-primJ- 
tije liquid-crystal video displsys 
used in moat helmet systems make 
the' views even rustier. For instance, 
Vlrtualty, a virtual reality game that 
already has appeared In aeveral 
shopping-mal video arcades around 
the nation. Including Springfield 
Mai, features cartoon-Ike charac- 



Some systems add a fifth compo- 
nent, an electronic wand or a "power 
glove"— en ela bo ra te array of hand- 
mounted electrical sensors— thst 
can be used to point at. grasp or 
move objects In the virtual world. 
The user can see the essctrorucalty 
generated hand or wand extending 
before Mm fa the TV smafay af the 
virtual world. 

It fa, for now, s cumber aoi nc set- 
up—especially when the thick cables 
used to connect It alt together are 
taken Into account* Reaearcheva are 



-4G 





M 3 » ** 



w C 4) 



or) 

cut: 
8 8 

^> ** s. 




.nig 

•6- "*£ 



er|c 



X 



3 



9 

ERIC 



1 

I 
S 



ii 

u 



m 



S „ «i g 2 

mil 



J. S3 A 

- IS 
1-1 

b " g 



J 



•a I -l -8 - 5 s 2 b - 

SflmJ 

s ^ -3 2 r 



I:. 



5 

•a 

c 

s 

8 



■a .1 



^ .: 



3 6 



".8 J=sa!i 



■35 



!;imliJ»! 



.41* 

J >..s 




RESEARCH 



The Scientist 
March 18, 1991 



Researchers See A Wealth Of AppUcations For Virtual Reality 

iwnk Thevir- Comouter Research and Techno 



By Robin Eisher 



X-ray cryptographer Vivian Cody 
has found a way to sit— virtually— 
in the midst of enxyme drag binding 
sites. She especially like* to experi- 
ence the feel of itall — the pushesand 
pulls that a drag goes through as it 
finds the coziest place to rest on a 
protein. Although this kind of mo- 
lecular space exploration soundslike 
fun and games, some scientists in 
acidemia, industry, and government 
see it as a most serious and advanced 
application of "virtual reality" an 
emerging computer technology that 
its most enthusiastic advocates sug- 
gest could change the entire com- 
plexion of research. 

While vinual reality has been 
used by the military and by space 
scientists for the last decade, phar- 
ma'joiogists, molecular biologists, 
and theoretical physicists are begin- 
ning to venture into its domain. Sim- 
ply speaking, the technology 
provides heightened representation 
of the real, physical attributes of sci- 
entific models. It is a tool that takes 
visualization and interpretation of 
data to a new dimension — to the 
point at which a researcher, in a 
sense, touches, interacts with, or is 
engulfed by the model that he or she 
has created. 

Unlike the usual interfaces be- 
tween humans and computers, such 
as the mouse or the keyboard, virtual 
reality hardware— head-mounted 
displays and gloves— allows people 
to feel as if they are entering the 
world inside a computer display, fly- 
ing around and manipulating the ob- 
jects they encounter (see accom- 
panying story). Indeed, it promises 
to accelerate scientific understand- 
ing by enabling researchers to refine 
their hypothesized models of the 
mvstenes of nature. 

'•^Science moves by its ability to 
see and models says Myron Krue- 
ger.oneoftl * innovators of the tech- 
nology. "Although we downplay our 
intuitive grasp of things and think 
mathematical equations are more 
correct, our intuition is more highly 
developed than our symbolic sense 
of the world. New perceptions and 
perspectives create radical sources 
f or d iscovery and insight in science." 

Other scientists agree. Crystal- 
lographer Cody, head of drug design 
at the Medical Research Foundation 
of Buffalo, is one of the small but 
growing number of academic, basic 
science researcher* — from biolo- 
gists to physicists— who have begun 
to use or investigate the use of VR in 
their work. Cody and other non- 
computer, nonspaoe. and noomili- 
tary scientists arc being introduced 
to VR through collaborations with 
private and public facilities apeciai- 
izmg in the technology. 

Cody felt the power of VR tech- 
nology last year at one of the centers 
uoing research on it m the Unhed 
States, the department of computer 
a t the Universitv of North 
v ia. Chapel Hill. ^ editor of 
£ RJ£ rmef ofMoteculmr Grtphia. 



Not yet, that is. The vir- 
tual landscape is expected 
to become more refined in 
five years. There are some 
50 VR startups, a number 
of major corporations, 
about five centers in the 
VS* and more in Japan 
doing research on VR. ac- 
cording to William 
Bricken, a scientist at the 
Human Interface Technol- 
ogy Laboratory in Seattle, 
one such center. But these 
firms and institutions 
aren't focusing on refining 
the technology for bench 
researchers. Most of the re- 
cent applications of virtual 
reality have been in render- 
ing 3-D architectural draw- 
ings, landing astronauts on 
| Venusian landscapes, med- 
ical imaging, and telecom- 
munications. Academic 

I scientists make up a small 

TSSJ?Sn^a^^^^^ [KKS^EE 



Cody says. "I often get to be 
the first kid on the block to 
play with new toys like 
[these]." And because the 
facility is considered a spe- 
cial resource center, funded 
by the National Institutes of 
Health, pan of the $300,000 
a year it gets must go to- 
ward collaborative work. 

Cody went to the center 
10 see if the university's 
prototypic VR force-feed- 
back, drug-docking pro- 
gram could reproduce 
results she gets with the cur- 
rent standard computer 
software and hardware. She 
left with a lot more. She 
found new binding orienta- 
tions for one of the drugs 
she works with— the en- 
zvme dihvdrofoUte reduc- 
tase (DHFR). a major 
chemotherapy target of 
anti-cancer agents in cells. 
Now she intends to deter- 
mine if what she saw in the 
virtual world can exist in the 

real world by crystallizing . , . . . 

tetouV^ZTt!*^'***" r«e. a person can ouickJy do by 



b etwee n a drug and in target 



configurations. 
Although the center has energy 
data available only for DHFR, the 
technology represents a major ad- 
vance over what is currently avail- 
able, Cody and others say. The 
fastest way to do all the energy mea- 
surcmentt between a drug and a sub- 
strate is to use a supercomputer," 
says Cody. But since the super- 
computer calculates every energy 
possibility between the atoms, it 
takes a tang time to do the analysis 
and is costly. VR putt a human being 
into the loop to both visualize and 
feel the interaction between the mol- 
ecules. The mind is still the beat 
pattern recognition system avail- 
able," says Cody. What the super- 
computer wastes time on doing by 



osing sight aad touch, she says. 

Although the technology prom- 
ises to be revolutionary, "there is still 
a trade-off between resolution and 
speed of transmission of images," 
says Michael McGreevy. head of the 
National Aeronautics and Space 
Administration s Virtual Environ- 
ment Research Laboratory at Ames 
Research Center in South San Fran- 
cisco, Calif. At the current level of 
commercially available virtual real- 
ity, for example, a biologist, could 
develop a highly stylized model of a 
cell and poke around inside. But the 
technology at the centers and on the 
market has not been refined to the 
point at which a biologist can enter 
into an image of a living cell and 
watch the Uuumutomcs divide. 



fore don't provide a sub- 
stantial economic incen- 
tive for product development 

But according to NIH computer 
specialist Richard Feldmann and 
others, designing affordable, mass- 
market consumer products, such as 
virtual video games or virtual tele- 
communication systems, are driving 
the VR marketplace. In such appli- 
cations, people would be able to 
"walk" into a computer game, or two 
people separated by distance could 
-occupy" a computer-generated 
space and interact in it. 

Spin-offs from these consumer 
products and advances in super- 
computers should eventually de- 
crease the coat of VR s oftware and 
hardware, say computer experts. And 
then scientists can benefit "Scien- 
tists are parasites in that sense." says 
Feldmann, of NIH's Division of 



WINDOWS ON THE VIRTUAL WORLD: 
HEAD-MOUNTED DISPLAYS 
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world Each semen has a 
diff cien t view so you see 
a stusoaconir, wada- 
angle view of thai com- 
nosrr aiidaaaJ place.* 

me head- 
_joauad daapiay," tft- 
biactt font on, •* a 
device caiksda*iiuisHi 
that measures year head 
position aad can dated 
where your tyaa* m* 
look* m apace. Ttt 
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stuff that was la front g 
off to the aide." 

Such lasad-moumed 
dsspiays were conceived 
an IKS aad created by 
bran Seeherland at the 
University of Utah in 
19tt; s l U aaar chrri are 
wortiatg on deveiopinf 
softwar and hardware so 
taw can ate images from 
the real world white ooor 
aria* m the virtual world. 
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Computer Research and Technol- 
ogy. In five to 10 years, say the ex- 
perts, virtual experimentation on a 
wider scale could become a reality. 

The computer graphics and vir- 
tual reality center at North Carolina, 
under principal investigator Fred 
Brooks, has been evolving since the 
early 1970s. Years of support from 
NIH. the Office of Naval Research, 
the university, and Burroughs Well- 
come Co. of Research Triangle Park. 
N.C, have finally begun to pay off. 
The jforce-feedback] drug docking 
project allows a researcher to feel in 
his or her hands the electrostatic 
forces that exist between a drug and 
ru substrate — in real lime. No small 
task." says Richard Dullois. director 
of the computer technology section 
of biomedical research in NlH's Na- | 
ttonal Center for Research Re- * 
sources in Bcthesda. MU. * 
To dock drugs at the North (aro- a 
Una center, the researcher stand> in | 
front of a 4* x 3* screen, upon which 
is displayed a model of a drug mol- 
ecule and its target protein. By using 
a robotic arm that acts as a joystick, 
he or she can move the drug up and 
down, left and right, and backward 
and forward. An energy histogram 
displayed on the screen allows the 
investigator to eyeball whether the 
fit between the drug and its substrate 
is energetically stable. At the same 
time, the robotic arm responds to 
repulsive forces between the mole 
cules depicted on the screen. 

Other projects at the center, in- 
cluding the drug-docking program 
without force feedback, involve one 
of the more popular interlaces of VR 
technology, called the head- 
mounted display. Force-fccdback 
drug docking using head-mounted 
displays is not possible yet. 'The 
technology is just not here.** says 
Walter Robinett chief of the head- 
mounted display project. But he and 
other; z~ working on it. 

Head-mounted displays are the 
window into the virtual world. The> 
create a stereoscopic, wide-angle 
view of the virtual world that the 
wearer can seemingly enter. Inside 
the displays are two small televi- 
sions, one for each eye. The view 
comes from a computer that gener- 
ates images 15 to 30 times per sec- 
ond. The picture changes depending 
on the wearer's movements. 

An input device allows a person 
to be transported inside the virtual 
world. These devices enable a per- 
son u> make commands that :he com- 
puter interprets. The computer then 
changes the image that it sends to the 
television screens in the head- 
mounted display. The input device is 
a joystick that has been trartsfot mcd 
into a glove that a person wean on 
the hand or a joyball with buttons. 

The glove was originally made 
by computer scientist Tom Zimmer- 
man because he wanted to play "air 
guitar"— actually producing the 
sound of strings with a flick of the 
wrist. He licensed the glove to iarc. 
Lanier, CEO of VPL Research Inc. 



Scientists Explore Uses Of Virtual Reality 
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, one of the 



in Redwood City. Calif. 
50 or so Yll startups. 

Wide acceptance and application 
of VR technology isn't going to hap- 
pen overnight Whik there ait tome 
commercial VR aoftwait and hard* 
ware products on the market, the ma- 
chines arc costly, and only the 
centers today have enough comput - 
ing power and expertise to create a 
believable, three-dimensional visual 
reality in which a researcher can in- 
teract. Before VR can be used mote 
widely, observers agree, a consider- 
able number of technical and finan- 
cial hurdles must be overcome. 

One criticism of virtual reality as 
it exists today is that the essential 
equipment is uncomfortable and un- 
reliable. The head-mounted dis- 
plays, for instance, are said to 
nauseate people. 

"The problem with the giove is 
that it is made of stretchy material, 
and it is hard to get it to work from 
day to day vith different sized 
nands,~ says Robin en. who uses the 
hall device. "But from the point of 
view of seeing a live hand in an 
environment and seeing the scale of 
things inside a virtual world in pro- 
portion to your own body, the glove 
is wonderful." 

An alternative to virtual reality, 
called artificial reality, overcomes 
some of the equipment unfriendli- 
ness. Invented by Myron Krueger in 
I WW. it b a system that allows a 
human being to interact in computer 
events unencumbered by gloves and 
hc;Kl-ut««mlcd displays. In Kruc- 
prr's artificial real it v. video cameras 



VPL Research Inc's 
Lasko: "Many people 
underestimate the 
power of the technology 
that does exist** 



send images of a person to s com- 
puter, which translates his or her 
movements and responds to them. 

One of the fastest-growing man- 
ufacturers in the U.S. of gloves and 
other virtual reality hardware and 
software is Redwood City 's VPL Re- 
search. The firm was founded in 
1985 by its president. Jean-Jacques 
Grimaud, and CEO Lanier, who has 
become a media celebrity of sons, 
having been profiled in Playboy* 
Rolling Stone, arxi Smithsonian. In 
June 1 989. VPL performed a lour de 
force— the first commercial demon- 
stration of two people in separate 
local ions sharing a virtual reality ei- 
pe Hence. 

Today VFL's customers induce 
telecommunications firms, engi- 
neering designers, and forestry re- 
searchers. And while VfLemc4oyees 
acknowledge that the technology at 
still in its early stages, customers say 
that it k worth getting in on the 
ground floor to gain an edge on what 
they predict will become a widely 
used technology m business. 

O e we started aelling our 



products, the only people who could 
work in this field had lo have all of 
the skills and resources to build the 
system from the grouad up," says 
Anne Lasko, director of product de- 
sign at VPL. They needed an opti- 
cal engineer to develop the eye 



phones and an electrical engineer to 
figure out the tracking. The system 
we sell is ready to go, so someone 
can make their own virtual reality. 

"But there are mis c onceptions 
about virtual reality. On the one 
hand, people have greater expecta- 



tions of the technology than is cur- 
rently available. On the other hand, 
many people underestimate the 
power of the technology that does 
exist. It is a very creative software. 
It is up to the user to see its applica- 
tion to their system.** 

VPL sells a virtual reality system 
for one person first includes a head- 
mounted display, the glove, the two 
advanced graphic workstations, and 



i for about SBXMMr. The 
two workstations account for 
$150,000 of the price. The glove 
alone goes for S&B00. 

Most academic researchers 
would be hard-pressed to find 
money from their university or from 
a federal agency to use VR in their 
lab, some observers say. ~lf a lone 
researcher, independent of the 
groups doing this son of thing, put in 




"...a powerful 
new tool for 
protein purification!" 



RF3: preparative isoelectric focusing rn 
a xfmsoiutioa No gees ex screen separators. Fast 
££*ihd easy to use. Processes 1 mg to 2 gmofpro- 
lain per two hour run 'tolerates pnxaprtauoa 
Integrated thirty tube fraction cotector High 
resolution and unique separation parameter 
Ccrnpiementstradrtior^ 
inaproiampurificatioam^ 
final product The RF3— a powerful new dimen- 
sion tor prolan purification. 
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g grant to the government even to do 
collaborative VR, it would be imps*- 
."lc to get it peer-fevicwed.*" says 
NlH computer specialist Fcidmann. 
"There just isn't enough of a base of 
scientists with expertise in applica- 
tions of thistechnoJofy toncw areas. 

"Peer review is a control mecha- 
nism that is somewhat repressive 
and reactionary. But once you get 
some bnve souls who show the tech- 



nology isn't just whir-bang, but can 
produce new and interesting results, 
'you migfc get some funding. This 
has happened before with other tech- 
nologies that people originally 
thought were just toys." 

Right now. in fact, the Ifi-rnooth- 
otd Human Interface Technology 
(HIT) Laboratory in Seattle is in the 
process of convincing the National 
Science Foundation to help fund its 



program. With Tom Fumess at its 
helm, getting money should be no 
problem, say colleagues. Fumess 
used to head the Dayton. Ohio-based 
Wright Paterson Air Force Base VR 
flight simulation project, a VR appli- 
cation used to train pilots. 

"HIT is a state-funded institution 
whose purpose is to transfer technol- 
ogy from the states' universities to 
the private sector,** says Bob Jacob- 



son, associate director of the center, 
based at the University of Washing- 
ton. HIT is also soliciting $50.<*I0- 
a-year memberships in a consortium 
program from industry investors. 
The pooled money from the mem- 
bers of the consortium supports gen- 
eral research in the lab. In return, 
industry members gain an advantage 
over competitors, since they have 
access to the initial stages of the VR 
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technology. Current irwmht s of t> s 
consortium include Digital Euurp- 
mcnt Corp. of Maynard. Mass.. Sun 
Microsystems of Mountain View. 
Calif., and VPL Research. 

For those academic researchers 
who want to ride the VR train but 
cannot afford it. collaboration with 
industry may be the ticket. John 
Joannopoulus, a theoretical physi- 
cist at the Massachusetts Institute of 
Technology in Cambridge, has such 
an arrangement with IBM*s Veridi- 
cal Uboratory in Yorktown Heights. 
N.Y. Hoping eventually to use VRas 
a visualization tool. Joannopoulus 
and colleagues arc in the process of 
developing a machine that could 
simulate the structure of solid*. Thi< 
machine would create thrcc-dimen- 



NIH'sFeldmann: M lfa 
lone researcher put in a 
grant (to use VRl.it 
would be impossible to 
get it peer-reviewed." 
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SMinaLclcctron-dcnNH> mansol sol- 
ids upon being provided only with 
the atomic number ol an clement. 

-Now I do the calculations on the 
supercomputers at MIT and Carne- 
gie Mellon Univcrsiu." he sa\s 
-Office of Naval Research mone> 
pays for them. I"hc \ isuali/.aiions of 
the solid structures I do at IBM." In 
the near future, he wants to he able 
to enter into the structure of a mm* id 
and sec whai an electron sce> sv ith 
VR. By disturbing the structure ol a 
pure solid with a contaminant, he 
wants Hi *ec the cflcct In*" iiMtfe 
the solid. 

But with all the money in the 
world Joshua Zimmcrhcrg Mill 
couldn't do * hat he wains to in vir- 
tual rcaliiv.ZimnicilKrg.cltict ol the 
Uboratory of Theoretical and Phys- 
ical Biology at the National Institute 
of Child Health and Human Devel- 
opment, wanted to take his t\\i*-tli- 
mensional fluorescent microscope 
images, translate them into a three- 
dimensional volume, and look at a 
cell as a whole, instead of section- 
ally. "We would need a supercompu- 
ter that doesn't exist yet to allow me 
to sec the cell in 3-D through a head- 
mounted display," says Zimmer- 
berg. who studies the hkiphysics ol 
nwrnbranc lusion. 'At the current 
time. VR is only go»*J lor people 
who work w ith computer-ucneraied 
images. But to do the kind ol work 1 
and others want to do is a good rea- 
son to push supercomputer tfcvclop- 
ment" 

Maintaining U.S. compe*iti\e- 
ncss in supercomputers and virtual 
reality prompted Sen. Albert (Sore 
(D-Tctw.) to visit North Carolina 
this past January. Gore is concerned 
that the Japanese will exploit the VR 
market before the VS. His concern 
may he warranted. According to 
HIT'S Jacobson. the Japanese have 
10 times as manv facilities research- 
ing VR as the US. NIH's DuBois is 
concerned as well, tic says. -If »c 
want to stay at the forefront of this 
field, we must be committed to keep- 
ing our technical base so wc aren't 
scooped by the Japanese.** □ 
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fn^1°? C * roM "*'« RUM TariOT ««» robot .rm to ,.„„ tjmulated molecu(ar 
forct fi.ldt around eomputor-gtnorotod imag.t of protiini. Story on Pig. A6 



"WWfMMO* 



50 



ERIC 



A6 • THE CHRONICLE OF HICHEft EDUCATION • ftlafcb IS, Ittl 



Computer-Created World of Virtual Reality 3 OpeningNew Vistas to Scientists 



By DAVID U WHtXXSM 

Computer-generated environments that 
create the ilktsiofi of three-dimensional 
worlds are pving scientists the ability to 
work surrounded by inut«3 of molecules 
and other objects that they once had to use 
an electron microscope to study. 

"You can take a stroll on an insect's eye 
or take a tour of an integrated circuit.'* 
say* Thomas A. Furness. director of the 
Human Interface Technology Laboratory 
at the University of Washington. His lab- 
oratory hopes to bring "virtual realities.** 
as the computer-created worids are often 
called, into broad public and scientific use. 

In the simplest version of the virtual 
worlds, computers generate images on 
goggles worn by the users, creating the 
sensation they are in a different place. 
Those developing the virtual realities hope 
they will open up new intellectual vistas in 
education, entertainment, art. and archi- 
tecture, as well as in science. 

William Brickcn. the chief scientist at 
the University of Washington laboratory, 
says that the primary purpose of virtual 
realities should be to go to "places that are 
absolutely unreal.*' 

Mr. Broken believes virtual realities 
could let people experience mathematical 
equations, for instance. Peopk could act 
as a variable in an equation and watch 
forms, colors, or curves shift around them 
in response to changes in the variable's 
value. 

Uftcd in Cancer Therapy 

At the University of Noah Carolina at 
Chapel Kill, variations on virtual-worlds 
technology have been used to help physi- 
cians position the beams of radiation used 
in cancer therapy and to aid biochemists 
searching for targets on protein molecule* 
where they can attach drugs. 

Virtual worlds may also inform the re- 
search of psycholog i sts who study percep- 
tion. Psychology, says. Mr. Brie ken "is 
the physics of virtual reality 

Computer scientists sec virtual-worlds 
technology as a natural extension of their 
work. "For 25 years," says Henry Fuchs. 
a professor of computer science at the Uni- 
versity of North Carolina, "the major 
thrust in computer graphics has been to 
con vey a sense of reality so the uscfXannot 
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displays— the same technology used in lap- 



distinguish computer-simulated objects 
from real objects. 

Researchers at the University of Wash- 
ington want to find more natural ways than 
keyboards for people to communicate with 
computers, "la the past, human talent and 
flexibility has been subservient to a narrow 
capability of computing." says Mr. Brick- 
en. *' We need to reverse tnis. Now that we 
have very sophisticated computational 
abilities, we need to adapt them to the 
needs of the human." 

He adds that people also need to begin to 
get used to the idea of "wearing" comput- 
ers. 

In one currently available version of vir- 
tuaJ*realtty technology, a user puts on gog- 
gles that create images with liquid crystal 



top computer screens. 

The picture in front of each eye of the 
goggle wearer is slightly different, creating 
the illusion of three dimensions. An elec- 
tromagnetic field is generated around the 
user, and wires running from a magnet on 
the top of a helmet that the user wears lets 
the computer generating the virtual reality 
know where the person's head is in the 
electromagnetic field. 

Glome Wit* Ftber*Optic Cables 

As the user's head moves, the computer 
adjusts the view being projected in the gog- 
gjes to what the user would be seeing from 
the new stance inside the virtual reality. A 
person viewing a "virtual tree*' would sac 



its other side if he or she walked in a i 
circle and looked back. 

For help in steering themselves through 
virtual realities, users usually wear a tlove 
with fiber-optic cables along the fingers 
and thumb. The glove relays commands to 
the computer. A pointing gesture, for ex- 
ample, leu a user fly through the virtual 
reality and indicates the direction desired. 

Those who have experienced virtual re- 
ality say the illusions are successful, but 
they sometimes find that the motion of im- 
ages are jerky and «he computer-generated 
objects look more like cartoons than the 
real thing. Experts in computer graphics 
who have seen the photo-realism possible 
on the screens of highTesolution. color 
monitors are generally disappointed when 
they don virtual -reality goggiev 

"The resolution is terrible" says Mr 
Fuchs. 

Researchers are experimenting with 
adding three-dimensional sound and a 
sense of touch to the virtual worlds. Bio- 
chemists who use a "oaort " system at the 
University of North Carolina can sec mole* 
cules ahead of them in three-dimensional 
space and. through a hand grip, can get a 
crude sense of the physical forces that 
would be generated if the molecules were 
real. 

The chemists say that feeling the atomic 
attractions and repulsions while they are 
manipulating the illusionary molecules 
helps them find the best way to attach one 
molecule to the other. 

Along with trying to plunge users into 
complete virtual worlds, scientists are us- 
ing the virtual-reality technology to aug- 
ment the normal senses. Researchers are. 
for example, developing systems that let 
Continued on Page A 12 
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phyucssas see and touch a pattern 
■wrmeliy but also let them see. 
superwiposed act the patient, in- 
foruuumn from medical imaging 
equipment that shows what it un- 
der the skin. 

An Obscure Domain 

Virtual-reality research, once 
the domain of a few laboratories 
that labored in relative obscurity, 
has suddenly come into more wide- 
spread use. Two virtual* reality 
newsletters and the discipline s 
first scientific journal are starting 
up this year, and scientists say cor- 
poration* are expressing interest in 
paving for the research. 



The corporal*, hanariag. com- 
bined with co mm ercia lly available 
hardware g og gles and gloves— 
has made k easier (or more soen- 
tiais to sort experiments. 

At least one computer scien- 
tist — Ivan E. Sutherland envi- 
sioned what he called "the Ulti- 
mate Display** almost 25 years 
ago. In 1961. Mr. Sutherland, then 
at the University of Utah, pub- 
lished a paper describing the first 
helmet that could project images in 
front of the viewer's eyes. 

Since then, researchers in the 
Air Force, at the University of 
North Carolina, at the Massachu- 
setts Institute of Technology, and 
at the National Aeronautics and 



Space Artmmiiiiatiun's Ames Re- 
search Center in Mountain View. 
Cel.. have worked to perfect the 
lecnnotoexs that have resulted in 
the first virtual worms. 

Yeeuef and Chaotic 

VmuaJ-rcaiity research as a dis- 
tinct discipline is young and chaot- 
ic scientist* say. Terms and their 
de fi n it io n s have yet to be agreed 
on. and even the name of the field 
is in dispute. 

'"The term 'virtual environ- 
ments' better fits a field of scien- 
tific research.'* says David 
Zeltxer. an associate professor of 
computer graphics at the Massa- 
chusetts Institute of Technotogy s 



Madia Laboratory. -Virtual reality 
is an uaaitainahk goal, like artifi- 
cial iotclligtecf " 

Those interested in virtual 
worlds say they have not been hin- 
dered by the lack of a journal. Us- 
ing ekctronic mail and specialized 
branches of computer networks in- 
tended for the builders of virtual 
worlds, researchers have been en- 
ergetically conversing and ex- 
changing bibiiographies. philoso- 
phy, references, and relevant com- 
puter programs. One scientist 
printed out the virtual-reality dis- 
cussions from one computer net- 
work and found he had $00 pages of 
reading ahead of him. 

Although some companies arc 
marketing hardware for creating 
virtual realities, the conversation* 
on the computer networks arc 



chiefly remind to research. "Any- 
one who gets into too much of a 
sales pitch is chastised/' says one 
software engineer. 

Research on "virtual realities'* 
has three chief components, says 
Mr. Broken. One pan consists of 
the effort to develop and improve 
technologies that make the com- 
puter-generated worlds look, 
sound, and feel more real. 

In another pan. researchers are 
writing computer pro g ra ms to or- 
ganize the vast amounts of data 
that stream back and forth between 
the people and the computers cre- 
ating virtual worlds. 

In the third, scientists are trying 
to improve their understanding <*f 
human perception so thev can do a 
better job of tricking the senses. 

"We're trying to tie these three 
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parts together as tightly as possi- 
ble.** Mr. Brickcnsays. 

The Birth mi Cyberspace' 

As the concept of virtual worlds 
spreads. Mr. fine ken and others 
believe it will be accompanied by a 
new view of electronically stored 
and communicated irtformatioo. 

"In the space around me. there 
are a dozen or so television chan- 
nels and hundreds of radio chan- 
nels jnd telephone conversations 
on cellular phones." V.r. Bncken 
says 

"Perceptually they're invisi- 
ble."' he adds, "yet given an instru- 
ment. I'm able to make them ap- 
parent. What's the word for that 
place '" 

That place has come to be called 
"cyberspace." a term lifted from a 



science-fiction novel that de* 
scribed a common hallucination 
sustained electronically. 

The concept of cyberspace — an 
electronic dimension separate from 
the physical world— has enthralled 
anthropotcgists. sociologists, phi- 
losophers, and artists. Michael 
Benedikt. a professor of architec- 
ture at the University of Texas at 
Austin, organized the First Annual 
Conference on Cyberspace last 
spring. It drew hardware builders 
and software writers as well as so- 
cial scientists. Mr. Benedikt says 
he came to appreciate the concept 
of cyberspace after a long period of 
ambivalence about the influence of 
ele ctr onic communication on 
chit court. But as he learned about 
cyberspace and virtual realities. 
Mr. Benedikt says he became con- 



vinced that they would open up an 
important new frontier for design. 

"I became intrigued by the idea 
that arejutccture could exist in an 
unreal space." says Mr. Benedikt. 
who is editing a book called Cyber* 
sneer; First Steps that wilt be pub- 
lished later this year by hit Press. 

This year s cyberspace confer- 
ence is being organized by Alloc- 
ouere Rosanne Stone, a lecturer in 
sociology at the University of Cali- 
fornia at San Diego. Ms. Stone sees 
virtual worlds as a more elaborate 
version of other electronic media 
tfret arc used socially, such as com- 
piler bulletin boards and tete- 
raone sex services. 

Ms. Stone says that virtual reali- 
ty. Eke ths other electronic media 
before it. carries the danger that 
people wiU become too engrossed 



in the cy ber sp ace image they pro- 
ject. "No matter how beauuful a 
body you code for yourself in high 
resolution. 3-D coior— no matter 
how seductive a space you enter." 
Ms. Stone says, "it will not slow 
down by one second how quickly 
you will die of aids if you have it. ** 

Limited and Expensive 

For now. access to virtual reality 
is limited and very expensive. The 
cost of start-up equipment alone is 
about S300.000 to (400.000. and se- 
rious researchers will need to 
spend more. 

Virtual-reality researchers even- 
tually hope to use telephone lines 
to allow users to enter virtual 
worlds even when they are away 
from the powerful computers need- 
ed to generate the worlds. Mr. Fur- 



aess envisions a "virtual com- 
mon"— a three-dimensional illu- 
sionary piece where participants, 
in their "virtual bodies." could 
mingle, converse, and listen to lec- 
tures or view entertainment. Mr. 
Fumes*, who spent 23 years in the 
Air Force building "virtual cock- 
pits.** has managed to interest at 
least one telephone company— 
U.S. West Comrruirucaions — in 
reuued research. 

Difficult to Focus 

Nine other companies, including 
atat Network Systems. Digital 
Equipment Corporation, and Sun 
Microsystems, are charter mem- 
bers of a consortium that Mr. Fur- 
aess has set up to further virtual- 
reality research. The corporations' 
chief concern seems to be that the 



technology has so manv apooca- 
ttons that it will be difficult for re- 
seaxeners to focus their efforts. 

Many obstacles block realizing 
the potential of virtual worlds. 
Massi\e electronic "pipelines" 
and fast switches will be needed to 
carry an enlarged flow of data be- 



"In the past, human 
talem and flexibility 
hat been eubaonrlent 
to a narrow capability 
of computing. We need 
to reverse tfita." 



rween people and computers. The 
University of Washington re- 
searchers estimate that the equiva- 
lent of 15.625 circuits capable of 
carrying the human voice will be 
needed to carry information about 
the position of a person's entire 
body to the computer and to bring 
iuJbrmauon back to that person s 
senses of sight, touch, and sound. 

At the University of North Caro- 
line. Mr. Fuchs says generating the 
three-dimensional images tor virtu- 
al worlds cats up so much comput- 
er power that the researchers have 
to search for ways to economize. 
One way to cut down is to lower 
the quality of the image a person 
sees when his or her head is mov- 
ing rapidly, but to increase the im- 
age quality when the head is still 



Another de vemp nw n ta l barrier, daw kand of freedom, 
says Mr. Facta, ss the cavrent As wrtssnl reality emiapment be- 
earthed of tracking head possuou. coaaaa snanistr. tighter, and cheep- 
la eied mgiiiiii tracking, users cr, its aeca aril spread, scientists 

have to stay within the conrmea of ear. 

a saaail elartmcnagiwiir held. **| hone to eve long enough to 

Scientists at tfce University of see the Walkman version of this." 
North Carohna wontd tike to try says Mr. Pucsss. ■ 
apeiicstwms that would give users 
e fr eedom of aai ■aenr. They 
£Ukei 
to be abie to walk i 
door sue w ear in g a saunas free of 
any wired connection to the conv 
pueer. Tim architects would be abet 
as see an imafc of the maiding they 
are pmnnine and the sac's fern urea, 
such as trees, at the same time. 

Mr. Pucsss and hmcottasauas are 
saapsnsang to cream an ocxksU 
trackiat system that wuuid allow 
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